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1. Introduction 



The main objective of this paper is to look from the unique point of view at some phenomena arising 
in different areas of probability theory and mathematical statistics. We will try to understand what 
is common between classical probabilistic results, such as the law of iterated logarithm for example, 
and well-known problem in adaptive estimation called price to pay for adaptation. Why exists 
two different kinds of this price? What relates exponential inequalities for M-estimators, so-called 
uniform-in-bandwidth consistency in density or regression model and the bounds for modulus of 
continuity of gaussian random functions defined on a metric space equipped with doubling measure? 

It turned out that all these and many others problems can be reduced to following one. Let T 
be a set and let (0, *B, P) be a complete probability space. Let x defined on T x be a given *B- 
measurable map into linear metric space © and let ^' : (3 — )■ M+ be a given continuous sub-additive 
functional. Let C T and suppose that we for any 9 £ Q and z > one can find U {9, z) such that 

V{[^{xe)-U{9,z)]>Q}<ce-\ c> 0. (1.1) 

Assuming additionally that XU (•, z) >U (•, \z) for any z > 0, A > 1 we also have for any ^ > 1 

E { [^>{xe) - U{9, z)]^y < cTiq + 1) [U{9, 1)] Vz > 1, (1.2) 

where T is gamma- function and [a]+ is a positive part of a. 

The problem which we address now consists in a finding of U(^, z) and U{9, z) satisfying 

p|sup[*(X0) -U(0,z)] > ol < ce"^; Vz > 1 (1.3) 
Uee J 

2-^ vz > 1, (1.4) 



e/sup [^{xe)-U,{9,z)\X <c, 

I 060 J 



inft/,«,,l) 



where c and Cg are numerical constants. If (1.3) and (1.4) hold we will say that U(-, •) and [/g(-, •) 
are upper functions for the collection of random variables {^{xe)i ^ S 0}. 
The main questions on which we would like to answer are the following. 

• Do U(-, •) and Uq{-, •) coincide with [/(•, •) up to numerical constants or there is a "price to 
pay" for passing from pointwise ( results 1.1)-(1.2) to uniform ones given in (1.3)-(1.4)? 

• Do U(-, •) and Uq{-^ •) coincide up to numerical constants? In other words should one to pay 
the same price for the probability and moment's bounds? 
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We will show that a payment exists and in general Uq{-,-) S> U(-,-) » U {■,■). Thus, we will 
seek U(-, •) and Uq{-^ •) satisfying (1.3) and (1.4) and "minimally" separated away from U{-, ■). We 
will realize this program under the following condition. 

Assumption 1. 1. There exist A : T — )■ M+, B : 1 ^ IR+ and c> such that Vz > 

^mxe)>z}<ce.p[-^,^^^^^], ye^Q. (1.5) 

2. There exist a : T x T — ;> M+ and b : T x T M+ such that Vz > 

P - X.. ) > 4 < c exp { - ^^^^^^J . Ve„«,6a (1.6) 

Remark 1. If Assumption 1(1) holds on T (not only onQ), T is linear space and if, additionally, 
the map xt is linear on T , then the Assumption 1 (2) is automatically fulfilled since one can take 
a(ti,t2) = A(ti -t2) and b(ti, t2) = S(ti -ta), ti,t2GT. 

Remark 2. We can easily deduce from (1-5) that for any 8 £ Q 

p{^{xe)>A{9)^/I+B{e)z'j<cexp{-z}, Vz > 0; (1.7) 
Ei^^{xe)-[A{e)V^ + B{9)zjy_^<cT{q + l)^A{e) + B{e)Y exp{-z}, Vz > 1. (1.8) 

Therefore, (1.1)^(1.2) hold with U{6,z) = A{6)y/l + B{9)z. 

Assumption 1 is not new. In particular, it can be found in slightly different form in van der Vaart and Wellner 
(1996), Talagrand (2005), where this assumption is used for deriving the bound for E [supgg0 ^'(xe)]- 
The usual technique is based on the chaining argument available in view of (1.6). It is worth 
mentioning that uniform probability and moment bounds for [sup5ig0 ^(xe)] in the case where 
Xe is empirical or gaussian process are a subject of vast literature, see, e.g., Alexander (1984), 
Talagrand (1994), Lifshits (1995), van der Vaart and Wellner (1996), van de Geer (2000), Massart 
(2000), Bousquet (2002), Gine and Koltchinskii (2006) among many others. Such bounds play an 
important role in establishing the laws of iterative logarithm and central limit theorems [see, e.g., 
Alexander (1984) and Gine and Zinn (1984)]. 

However much less attention was paid to finding of upper functions. The majority of the papers, 
where such problems are considered, contains asymptotical results, see, i.e. Kalinauskaite (1966), 
Quails and Watanabe (1972), Bobkov (1988), Shiryaev et al. (2002) and references therein. We 
would like especially mention the paper Egishyants and Ostrovskii (1996), where upper function 
satisfying the inequalities similar to (1.3), was obtained for the modulus of continuity of random 
fields satisfying the Cramer condition. 

The researches carried out in the present paper complete the investigations done in Goldenshluger and Lepski 
(2011), where the upper functions as well as inequalities (1.3)-(1.4) were obtained under following 
condition: xt is linear and there are A :% ^ R_|_, : T — t- M+, y : T — ?■ M+ such that 

PWx.)-m>4<»( ^.(,);^(t)J . vt.i, 

where g : M+ — )• M+ is a strictly decreasing to zero function. We note that if g{x) = e~^ and 
y = this assumption coincides with (1.5) and, since xt is linear (1.6) is automatically fulfilled, see 
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Remark 1. In Goldenshluger and Lepski (2011) under additional assumption imposed on A,B,V 
and G C T the upper functions for the collection {^{xd)^ ^ £ ©} were found. As it was shown that 
they distinguish from the function V{-,-) only by numerical constants! The imposed assumptions 
do not admit the case V = that, as it was said above, leads to completely different solution of 
the problem at hand. 

To derive upper functions satisfying (1.3)-(1.4) we complete Assumption 1 by the following 
conditions. 

Assumption 2. x» '■ ^ ^ ^•s continuous F-a.s. 

Mappings a and b are semi-metrics on T and Q is totally bounded with respect to a V b. 

Aq := supgge M^) < oo, ^0 := sup^g© B{0) < oo. 

Denote by S the following set of real functions: 

S = |s :M^M+\{0} : ^ s(2^'/2) < l|. 

k=o ^ 

For any C and any semi-metric d on T let ^{^), S > 0, denote the entropy of Q measured 
in d. For any x > 0, C and s G S define the quantities 

e'f\x,e) = sup (5-2^0 {x{486)-^s{6)) , e(^)(x,0) =sup(5-^gs , (x{48S)-h{6)) .(1.9) 

Assumption 3. There exist si,S2 € S such that Vx > 

e(^)(x,0) <oo, eg)(x,0)<oo. 

Organization of the paper In Section 2 we construct upper functions for {^(xe)i & G ©} and 
prove for them the inequalities (1.3)-(1.4) under Assumptions 1-3. In fact we present two different 
constructions which will be refereed to upper functions of the first and second type (Propositions 2 
and 3). We also derive some consequences related to the upper functions for modulus of continuity 
of random real- valued mappings (Propositions 4 and 5 ). In Section 3 we apply Propositions 3 
and 4 to gaussian random functions. In Section 3.1 we derive upper functions for Lp-norm of 
some Wiener integrals (Theorem 1) and in Section 3.2 we study the local modulus of continuity of 
gaussian functions defined on a metric space satisfying doubling condition (Theorem 2). Section 4 
is devoted to the detailed consideration of generalized empirical processes. We provide with rather 
general assumption (Assumption 4) under which the upper functions admit the explicit expression. 
Section 4.2 (Theorem 3) and Section 4.3 (Theorem 5). We also establish non-asymptotical versions 
of the law of iterated logarithm (Theorem 4) and the law of logarithm (Theorem 6). Section 4.4 
is devoted to the application of Theorems 3 and 5 to empirical processes possessing some special 
structure. Theorems 7-10. Proofs of main results are given in Sections 5-7 and technical lemmas 
are proven in Appendix. 

2. General setting 

Denote by 5a,b the subset of S x S for which Assumption 3 holds and let A,B,sl and b be any 
mappings for which Assumption 1 is fulfilled. For any s = (si,S2) £ ^a^b, any x = (xi,X2), xi > 
0, X2 > 0, and any C put 

e^x, 0) = e(^) (xi, 0) + ei^) (x2, 0) . (2.1) 
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2.1. Inequalities for the suprema 



Put for any B C 0, any e > and any y > 

U^'\y,x,e) = XI ^2 [1 + £-1] \g{x, e) + y + X2 (2 [1 + e^^] \g{x, O) + y 



Proposition 1. Let Assumptions 1-3 hold and let Q Q be fixed. Then for any k satisfying 
Tk\ > supggQ A{9) and X2 > sup^gg B{9), any s G 5a,b; £ G (O, \/2 — l] and y > I, 

p\snp^{X9)>ui'\y,^,e) I <2cexp{-y/(l + e)2}. 



Moreover, for any q > 1 
( 

Tie) 



E<'sup^(xe)-C/f (y,x,e) !> <2cT{q + l) {I + efy-^uf [y^x.Q) ' exp {-y/(l + e)^}. 
Uee J + 

We remark that sup^^Q ^ {xe) is ^-measurable for any C since ^ is continuous, the mapping 
I— 7- is continuous P-a.s., is a totally bounded set and considered probability space is complete 
(see, e.g. Lemma 1 below). 

Discussion We will see that the Proposition 1 is crucial technical tool for deriving upper func- 
tions. It contains the main ingredient of our future construction the quantity e^(-, •). The important 
issue in this context is the choice of s G ^a^b- For many particular problems it is sufficient to choose 
s = (s*, s*), where 

s*{x) = (6/7r2)(l + [Inxfy^, x > 0. (2.2) 

This choice is explained by two simple reasons: its explicit description allowing to compute the 
quantity eg in particular problem and the logarithmical decay of this function when x — )■ 00. In 
view of the latter remark we can consider the set whose entropy obeys the restriction which is 
closer to the minimal one (c.f. Sudakov lower bound for gaussian random functions Lifshits (1995)). 
We note, however, that there exist examples where s has to be chosen on more special way (see 
Theorem 1). 

Let us now discuss the role of parameter e. In most particular problems considered in the paper 
we will not be interested in optimization of the numerical constants involved in the description 
of upper functions. If so, the choice of this parameter can be done in arbitrary way and we will 
put e = \pl — 1 to simplify the notations and computations. Note, however, that there are some 
problems (see, for instance Section 3.2), where e must be chosen carefully. The typical requirements 
to this choice is e = e(y) and 

e{y) 0, yE^{y) ^0, y ^ oo. 

The bounds similar to whose presented in Proposition 1 are the subject of vast literature see, for 
instance, the books Lifshits (1995), van der Vaart and Wellner (1996) or van de Geer (2000). Note, 
however, that the results presented in the proposition may have an independent interest, at least, 
for the problems where the quantity e^(-, •) can be expressed explicitly. In this case under rather 
general conditions it is possible, putting = and Tk = (Ae, -Be), to compute the tail probability 
as well as the expected value of the suprema of random mappings. Note also that Assumptions 1-3 
guaranty that EjsupggQ ^ (xe)}"^ is finite for any g > 1. 
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2.2. Upper functions of the first and second type 



We will now use Proposition 1 in order to derive the upper functions for (xe) on 0. Denote 
A = infeee A{e) and B = inf^ee B{e). 

We present two kinds of upper functions for ^ (xe) on G which we will refer to upper functions 
of the first and second type. The first construction is completely determined by the functions A, 
B and by the semi-metrics a and b. It requires however the additional condition A> Q,B_> {). We 
will use corresponding results for the particular problems studied in Section 4. 

The second construction is related to some special structure imposed on the set 0. Namely we 
will suppose that = Uo-Qagsi, where {©«, a G 21} is a given collection of sets. Here we will 
be interested in a finding of upper function for supggg,^ ^ {xe) on 21, which can be also viewed as 
an upper function for ^ {xe) on 0. The corresponding results are used in order to obtain rather 
precise inequalities for the modulus of continuity of random functions, Section 3.2. Moreover we 
apply this bound for deriving of an upper function for the Lp-norms of Wiener integrals, Section 
3.1. We deduce the corresponding inequality directly from Proposition 3 below without passing to 
the concentration inequalities. 

We finish this short introduction with the following remark. In order to establish the inequali- 
ties (1.3)-(1.4) for the upper functions presented below we will need to prove that corresponding 
supremum is a random variable. The result below is sufficient for all problems considered in the 
paper and until proofs we will not discuss the measurability issue. 

Lemma 1. Let T he the set equipped with the metric c), (fi,^,P) he a complete probahility space 
and ^ : f2 X T — 7- M 6e F-a.s. continuous. Let be a set, g : ^ M he a given function and 
{Tj ^ 2^, 3 G 3} &e o-n- arbitrary sequence of sets. IfT is totally hounded then sup^g^ s^PteTj 
5(3)] is -measurable. 

The proof of the lemma is given in Appendix. We would like to emphasize that there is no any 
assumption imposed on the function g, index set 3 and on the collection {T^ ^ 3 G 3}- 
Putting 3 = "J and Tt = {t} we come to the following consequence of Lemma 1. 

Corollary 1. Under assumptions of Lemma 1 sup^g^ [C(t) ") ~ ^(t)] 'is ^-measurable. 

Upper functions of the first type As it was said above throughout this section we will suppose 
that ^ > 0, ^ > 0. Put for any t > 

OAit) = 1^ G e : A{e) < i}, Qeit) = |^ G 9 : B{e) < i}. 

For any s G 5a,b introduce the function 

£siu,v) = eif(Au,GA{Au))+e^J^UBv,eB{Bv)), u,v>l. (2.3) 



Denote also i{u) = In {1 + In (n)} + 2 In {1 + In {1 + In (u)}} and set for any 9 and e > 0, r > 

PeiO) = 2[1 + e-'] ^£{Ae{e),Be{9)) + (1 + ef [i{Aeie)) + e{Be{e))] ; (2.4) 
M,^r{0) = (1 + e)' {2[1 + e-i] '£{As{e),Beie)) + {e + r) In [Asie)Bsie)] } , (2.5) 
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where Aei9) = (1 + e) [A{e) /A] and BeiO) = (1 + e) [B{e) /B] . Define for any z > 
V(-'^)(0) = (1 + ef (A{6)^Pe{e) + (1 + efz + B{6) \p,{e) + (1 + ef 



(2.6) 



In the proposition below we prove that the functions defined in (2.6) and (2.7) are upper functions 
for ^ {xe) on Q. We remark that the behavior of ^ (xe) on is completely determined by the 
functions A and B and by the entropies of their level sets measured in semi-metrics a and b. The 
number e and the couple of functions s can be viewed as tuning parameters allowing either to 
weaken assumptions or to obtain sharper bounds but they are not related to the random functional 
^ (xe) itself. 

Proposition 2. Let Assumptions 1-3 he fulfilled. Then Vs € 5a. b, Ve G (O, \/2 — l] and Vz > 1 



P \ sup 



^(X(?)-V(^'^)(^)' 



> ^ < 2c 



1 + 



ln{l + ln(l + e)} 



exp{-2;}; 



^(Xe)-U("'''''^(0)j i < c2(5'?/2)+2p(^ ^ ^-<?-4 [^vS]''exp{-z}. 



E < sup 
leee 

It is obvious that the assertions of the proposition remain valid if one replaces the function 
£ by any its upper bound. It is important since the exact computation of this function is too 
complicated in general. We note that the role of the latter function in our construction is similar 
to whose which Dudley integral plays in the computations of the expectation of the suprema of 
gaussian or sub-gaussian processes Lifshits (1995), Talagrand (2005). 

Price to pay for "uniformity" We remark that in view of (1.7) and (1.8), the function 
u^^\e) := A{e)^ + B{e)z can be viewed as "pointwise upper function" for "^{xe), i-e. for fixed 
6. Comparing the inequalities (1.7) and (1.8) with whose given in Proposition 2 we conclude that 
they differ from each other by numerical constants only. In this context, the functions P(-) and 
Mg(-) given by (2.4) and (2.5) can be viewed as price to pay for "uniformity". That means that in 
order to pass from "pointwise" result to the "uniform" one we need, roughly speaking, to multi- 
ply A{-) by P(-) or ^y Mq{-) and B{-) by P{-) or Mq{-). The question, arising naturally: is such 
payment necessary or minimal? We do not think that the answer can be done under "abstract 
considerations", i.e. under Assumptions 1-3. However, for particular problems it is seemed pos- 
sible. Unexpectedly the answer on the formulated above question can come from the solution of 
the problem studied in mathematical statistics. In this context it worths to mention the relation 
between well-known phenomenon in adaptive estimation, called price to pay for adaptation Lepski 
(1991), Lepski and Spokoiny (1997) and Spokoiny (1996), and what we call here price to pay for 
uniformity. We have no place here to describe this relation in detail and mention only several facts. 

First let us remark that Proposition 2 contains the results which can be directly used for 
the construction of adaptive procedures. Indeed, almost all constructions of adaptive estimators 
(model selection Barron et al. (1999), risk hull minimization Cavalier and Golubev (2006), Lepski 
method Lepski (1991), or recently developed universal estimation routine Goldenshluger and Lepski 
(2008, 2009)) involve the upper functions for stochastic objects of different kinds. Next, it is known 
that there are two types of price to pay for adaptation: (In)-price, Lepski (1991) and (In ln)-price. 
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Spokoiny (1996). The (In)-price appears in the problems where the risk of estimation procedures 
is described by a power loss-functions and it corresponds to the function Mg(-), where the param- 
eter g is a power. The (Inln)-price appears in the case of bounded losses that corresponds to the 
function P{-). Since the theory of adaptive estimation is equipped with very developed criteria 
of optimality, Lepski (1991), Tsybakov (1998), Kluchnikoff (2005), we might assert that the pay- 
ment for uniformity is optimal if the use of corresponding upper function leads to optimal adaptive 
estimators. 

We finish the discussion concerning the statements of Proposition 2 with the following remark. 
Comparing the result given in (1.8) with the second assertion of Proposition 2 we can state that 
the inequality obtained there is very precise since, remind, A = inf^ge A{9) and = infgge B{9). 

Upper functions of the second type Suppose that we are given by the collection {©a, a G 2t|, 
satisfying Q = [Ja^%Qa, and by two mappings ti : 21 — )■ (O, ri] , T2 : 21 — )• (O, , where ri, r2 < oo. 
For any n > put 



e;(n)= U e„, g\{u)= sup A{e)- 



a.: ri (ci)<ti 



eee'i(«) 



e'2(n)= IJ G„, g*B{n)= sup B{e), 

a:r2{a)<u ^^O'^M 

and let qa and qb be arbitrary chosen increasing functions, satisfying qa > Qa and qb > g% (we 
note that obviously g\ and g*B are increasing) . 

Since ©'i('))02(') — ®' view of Assumption 3 for any u,v > one can find the functions 
si{u, •) and S2{v, •) for which the latter assumption is fulfilled on Q[{u) and ©2(t') respectively. Let 
us suppose additionally that 

si(xt,6) S2(xt,6) 
Ai := sup sup sup — - — — < oo, A2 := sup sup sup — - — — < 00, (2-8) 

te[i,V2] ^>Li s>o si[x,d) te[i,V2] '■^>i.2 5>o S2{x,d) 

where = inf^ Ti(a) and t_2 = info, T2{a). 

We remark that if the functions si{u, •) and S2{v^ •) are chosen independently of u,v then Ai = 
A2 = 1. It is also obvious that Ai, A2 > 1. 

The condition (2.8) allows us to define the function: 

S'{u,v) = eif(,_.)(Ar^gA(n),e;(^.)) +e2J^,.)(A2i5B(«),e'2(^;)), u,v > 0. (2.9) 

We note that the function £' is constructed similarly to the function £ used in the previous section, 
but now the functions si and S2 can be chosen in accordance with considered level sets. 
At last, for any a G 21 and any e > set 

£^'\a)=£'({l + e)Tiia),il + e)T2ia: 



Put Sj = (1 + e) -',J > 0, and let : x M.^ — )• M+,r > 0, be an arbitrary family of increasing 
in both arguments functions, satisfying for any e € (O, \/2 — l] 

J K 

[9A{riSj) V gB{r2Sk)Y e^p{-Rr{Ti5j,T26k)} =: < 00. (2.10) 

j=0 k=0 



Here integers J, K are defined as follows. 

J = [lni+, (ri/ri)J +1, K= (^2/12)] + 1. 

If Tj = 0, i = 1, 2, the corresponding quantity is put equal to infinity. 
Set R^r\oi) = RrUl + e)ri(a), (1 + e)r2(a)) and define 



+ {l + efgB([l + e] V2(a)) (2 [l + e~^f S^'^ (a) + Ri'^ (a) + , 



Below we assert that ij('^'^'^)^r = 0,r = q, are upper functions for supgg@^ ^ (xe) on 2t. However, 
before to present exact statements, let us briefly discuss some possible choices of the functions Rr- 
We would like to emphasize that the opportunity to select these functions allows to obtain quite 
different and precise results. First possible choice is given by 



Rq{u,v) = i(TlU )+£(T2V 



Rr{u, v) = £ 



In ( Tiu ^] + In ( T2V 



, r > 0. 



(2.111 



These functions are used in the problems in which <?'-^-'(-) is bounded by some absolute constant 
independent of all quantities involved in the description of the problem, assumptions etc. 
This choice leads to the following values of the constants in (2.10): 



2 + 



ln{l + ln(l + e)} 



(2.12) 



Another important choice is given by Rr = E' independently of r, see, for instance. Theorem 1. In 
view of (2.10), this choice corresponds to the case when the function E' increases to infinity. 

Proposition 3. Let Assumptions 1-3 he fulfilled. Then for any s\^S2 satisfying (2.8) and any 
Rr, r > 0, satisfying (2.10), for any e G (O, \/2] and any 2 > 1, g > 1 



P \ sup 



E \ sup 



sup ^(Xe)-U(^'^'°)(a) 
See,, 



sup ^(Xe)-U^^'"''')(a) 



> o| < 2c7^(^'°)exp{-z}; 



Remark 3. We note that the results of the proposition is very general. Indeed, there are no as- 
sumptions imposed on the collection 0^, a G 21, and the functions ti,T2 can be chosen arbitrary. 
Moreover, the condition (2.10) is very mild, so the choice of functions Rr is quite flexible. 



2.3. Upper functions for the modulus of continuity of random mappings 

In this section we apply Proposition 3 in order to derive upper functions for the local and global 
modulus of continuity of real- valued random mappings. It is worth mentioning that in this circle of 
problems the upper functions are actively exploited, see e.g. Egishyants and Ostrovskii (1996) and 
the references therein. We will suppose that Assumption 1 [2), Assumption 2 and Assumption 3 
are verified, xt is real-valued random mapping defined on the metric space T, d is a semi-metric on 
T and ^(0 = | • |. 
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Upper function for local modulus of continuity Let be a fixed element of G and set for 

any A € (O, Dd(0)] > where L'd(0) is the diameter of © measured in the semi- metric d, 



mA(6'o 



snp\xe-Xeol ©A = G © : d(^, 0o) < a|. 

left A ■> 



0e0A 



Thus, mA(^o)5 A S (O, Z?d(©)] , is the local modulus of continuity of xe in measured in d. 

If we put Xe = XO — X0OJ ^ G ©, we assert first that Assumption I (2) can be viewed as Assumption 
1 (i) for Xe on © with A{-) = a(-, 6q) and B{-) = b(-, Oq). Next, noting that xei — Xe2 = Xei — X6»2 
for any ^1,6*2 G © we conclude that Assumption 1 (2) is verified for xe on © with a and b. 

Thus, we can apply Proposition 3 with a = A, ©^ = ©a, 21 = (0,Dd(©)] and we choose 
ri(A) = T2(A) = A. This choice implies obviously for any u < Dd(0) 

©i(m) = ©2(u) = ©«, 9Aiu) = sup a{0,9o), 9b{u) = sup b(6',6'o). 

e:d(6»,eo)<M 61: d(e,6»o)<« 

Fix s G 5a,b and put for any A G (O, Z)d(©)] and any e G (O, \/2 — l] 

f (A, ^0) = ei^) (gA{[l + e] A) , ©[i+,]a) + e^^) U ([1 + e] A) , ©[i+,]a) • 



Here efx and 6^2^ are defined by (1.9). We also set Ai = A2 = 1 since the functions si, S2 are chosen 
independently of the collection {©a, A G (O, Z)d(©)] }• 

Choose also Rq{u,v) = 1\tiu~^] + 1\T2V~^\ and define 



yj^'^) (A, Oo) = (1 + e)<7^ ([1 + a) y 2 [1 + e-i] (-) (A, ^0) + (1 + [2^ (l?d(e)/ a) + z 

+ (1 + ([1 + e]2 a) {2 [1 + 6-1] (A, ^o) + (1 + e)' [2£(Dd(©)/A) + } . 

Then, applying Proposition 3 and taking into account (2.12) we come to the following result. 
Proposition 4. Let Assumptions 1-3 he fulfilled. Then Vs G 5a, b. Ve > and Vz > 1 

on 2 



P < sup 

Ae(o,Dd(e) 



nxA 



y^^)(A,0o) 



> > < 4c 



2 + 



ln{l + ln(l + e)} 



exp {— z}. 



In Section 3 we apply Proposition 4 to gaussian random functions defined on a metric space 
satisfying so-called doubling condition. 

£^^\A,eo) =: S^'HOo) < 00, the upper function 



Remark 4. // b = 0, d = a and sup 

V^^^''^'* has very simple form 



Ae(o,Dd(e)] 



vj^''\A, 9o) = (1 + e)=^A. /4[1 + e-^Y£(^){9o) + (1 + e)^ 4^d(©)/A + 



(2.13) 

Hence, the result of Proposition 4 can be viewed as the non- asymptotical version of the law of 
iterated logarithm for sub-gaussian processes defined on some totaly bounded subset of metric space. 
In this context it is worth mentioning the paper Egishyants and Ostrovskii (1996) where the upper 
functions for local and global modulus of continuity were found for the stochastic processes satisfying 
Cramer's condition. 
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Remark 5. We also note that we replaced in (2.13) the factor 2l(^D^{Q) j appeared in the 
upper function used in Proposition 4 by ^(L'd(0)/'^) • It is explained by the fact that T2 = 
in (2.11) since B,h = 0. By the same reason, the probability bound in this case is given by 



4c 



2 + 



ln{l + ln(l + e)} 



-2 



exp {— z}. 



Upper function for global modulus of continuity Set 0^^^ =0x0 and let for any 'd = 

{61,92) G 0(2) and any A E (O,Z)d(0)], 

m = Xe,-Xe„ mA= sup |C^|, 0^^ = G 0^2) : d(0i,^2)<A|. 

Thus, rtiA, A G (0, l?d(0)]) is the global modulus of continuity of xe on measured in d. 

Put A(t9) = a{9i,e2), B(i9) = h{0i,e2), ^ = (^1,^2) G and equip ©(2) with the following 

semi-metrics: -i? = (0i,02),<? = (?i,?2) G Q^'^^ 



,(2) 



(i9,<^) = 2[a(0i,?i) Va(02,?2)], b(2)(^?, ^) = 2 [b(^i, <Ji) V b(^2, ^2)] 



Some remarks are in order. We note first that Assumption 1 (2) can be viewed as Assumption 1 
(1) for C(i?) on 0(2) with ^ = A and S = B. 

Next we obtain in view of Assumption 1 (2) W, <^ G ©^^^ and Vz > 

P{|CW - C(?)l >z}< P{|xe, - X.J > ^/2} + P{|xe2 -x,,\> z/2] 

< c exp < 7^ + c exp < ^ > 

i 4[a(^i,<ji)]' + 2b(ei,?i)zJ i 4[a(e2,?2)] +2b(02,?2)^J 



[a(2)(^,^)]'+b(2)(i?,(^)z J 

We conclude that Assumption 1 (S) holds for on 0^2) with (2) b = b(2) and c(2) = 2c. 
Since obviously 

£,(2),e(2)(0<2ea,e(?/2), ^b(2),e(2)(0 < 2e:b,e(?/2), ? > 0, (2.14) 

we assert that Assumptions 2 and 3 are fulfilled on 0(2) with a = a(2) and b = b(2). 

Put 0^^^ = Ua>o0a^- Since 0^^^ C ©(^^ we can apply Proposition 3 with a = A, 0^ = 
0(f\ 21 = (0,L>d(©)] and we choose ri(A) = T2(A) = A. 
The latter choice implies obviously for any u < Dd(0) 

@[{u) = 0'2(n) = 0(f), gj^{u) = sup A{^), <7B(n) = sup B{^). 
Fix s G 5a,b and put for any A G (O, Dd(0)] and any e G (0,^/2-1] 

(2) _ \^M'')(nr.(h ^plA^ «(2) 



^(^)(A) = e(f' ) (,4[1 + e]A), 0(?|^j^) + eg-) (,^[1 + e]A) , ©[J) ^j^) . 
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Here eif' ^ and eif 

are defined by (1.9), wliere a, b are replaced by a^^) and b^^) respectively. 
We also set Ai = A2 = 1 since the functions si,S2 are chosen independently of the collection 
A e (O,Dd(0)]}- Choose Ro{u,v) = £(tiu^^'^ +£(t2V-^^ and define 



F/'^^(A) = (1 + e)5^([l + e]'Aj ^2[1 + e~^Y£(-)iA) + (1 + e)^ [2^(Z)d(e)/Aj + 

+ (1 + efgB{[l + e]'A) {2[l + e-i]'f(^)(A) + (1 + ef [2^(Dd(e)/A) + z] } 

Then, applying Proposition 3 and taking into account (2.12) we come to the following result. 
Proposition 5. Let Assumptions 1-3 be fulfilled. Then Vs € 5a, b> Ve > and Vz > 1 



sup 

Ae(o,Dd{e) 



mA 



i>i^'^)(A) 



> > < 4c 



2 + 



ln{l + ln(l + e)} 



exp { — z}. 



The obtained inequality allows, in particular, to prove that the families of probabilities measures 
generated by xe is dense. This, in its turn, is crucial step in proving of the weak convergence of 
probabilities measures. 



3. Gaussian random functions 



In this section we apply Propositions 2-4 to the family of zero-mean gaussian random functions. 
Thus, let X01 ^ G 0, is a real valued continuous gaussian random function such that Exe = 0, G 
0. We are interested first in finding an upper function for |x6»|) ^ G ©. Let 

V{e) = ^Y.\xe\\ P{ei,d2) = ^'&\xe,-Xe,? 

We remark that Assumption 1 holds with c = 2, S = and b = and \/A > V^V, Va > V2p. 
Since b = Assumption 3 is reduced to 

Assumption 3 [Gaussian case]. There exist s G S such that for any x > 

sup(5^2eg {x{A86y^s{6)) < 00. 

<5>0 ' ^ 

Thus, if the latter assumption holds. Propositions 2-4 can be applied. 

The aim of this section is to find uppers functions for quite different functionals of various 
gaussian processes. We would like to emphasize that the original problem is not always related to 
the consideration of jxeji^ G 0) although such problems are also studied. The idea is to reduce it 
(if necessary) to whose for which one of Propositions 2-4 can be used. Without special mentionning 
we will always consider a separable modification of ^ G 0. 



3.1. Upper functions for hp-norms of Wiener integrals 



Let iiT : M'^ — )• M be a continuous compactly supported function such that ||-ftr||oo < 00. Without loss 
of generality we will assume that the support of K is [-1/2, 1/2]*^. Let < /i^'^''^) < /i^'^^^) < 1 be 



given numbers. Put Ti 



^(min) /j(max) 



and let Kh{- 
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h~'^K (-/h) , h gT-L. Here u/v, u,v £ 



denotes coordinate-wise division. Let b{dt) is white noise on and consider the family of gaussian 
random fields 

Chit) = h-'' [ Kh [t - u) 6(dn), h(en. 
Let K.^ = [— /i/2, ^ > 1, be a given cube and let for any 1 < p < oo 



The objective is to find an upper function for ||C/i||p on T-L and later on Ci,C2 ■ ■ ■ , denote the 
constants completely determined by d, p, fj,, 7 and K. It is worth mentioning that the explicit 
values of these constants can be found and some of them are given in the proof of the theorem. 
We will be interested only the case 2 < p < 00, since for any p e [1,2) we obviously have 

d{2-p) 

||a||,<(^) M, 

and, therefore, we can use the upper function found for p = 2 for any p G [1,2). 

Let B^ ,,,s > 0, 1 < g',r < 00, denote the Besov space on W^^ see e.g. Edmunds and Triebel 
(1996), and later on IH[q(s, L) denote the the ball of the radius L > in 

Suppose that K G IHIoo(7, -^^) and without loss of generality we assume that L = 1 that implies 
in particular that ||i^||oo ^ 1- 

Theorem 1. Assume that 7 > d/2. Then for any 2 < p < 00, 

/j(min)^;j(max) ^ (q^ and q > 1 

p{sup[||4||p-Ci/i-'^/2] >o} < C2exp|-2-3/2(/,(-x)^-2'^/^| 

e| sup [llallp - Cih-^/'] y < Gsiq) (/i^--))"^ exp |-2-3/2 ))"''^''|. 

The proof of the theorem is given in Section 6. The constant Ci involved in the description of 
found upper functions is bounded function of p on any bounded interval. Thus, the upper functions 
are independent of p if p G [2,po] for any given pQ > 2. 

Also it is important to mention that the obtained upper functions are sharp. Indeed, it is not 
difficult to prove that for any h > 

c^h-''/^ <E\m^<c5h-''/^. 

This, together with the concentration inequality for gaussian processes, Talagrand (1994), yields in 
particular for any given h > 



This inequality coincides, up to numerical constants, with the first inequality in Theorem 1 in 
particular case when = = h. 
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3.2. Upper functions for local modulus of continuity under doubling condition 

Let metric space (T, d) be equipped with Borel measure x. This measure is doubhng if 3Q > 1 
such that 

x{Bd(t,2r)} < Qx{Bd(t,r)}, Vt G T, Vr > 0, 

where lBd(t, r) is the closed ball with center t and radius r. For example, if T = M*^ and x is Lebesgue 
measure then Q = 2"^. 

As it was proved in Coifman and Weiss (1971) the existence of a doubling measure on T implies 
that the space T is doubling. It means that there exists A^d £ depending only on Q such that 
for any r > each closed ball in T of radius r can be covered by at most Ad closed balls of radius 
r/2. This yields that IB(t, r) is totally bounded for any t € T, r > and, moreover, 

e^B,(t,r),d(5) < In(A^d) ([log2 {r/5}]^ + l) , V5 > 0. 

Let t € T and r > be fixed. In this section, using Proposition 4 we establish the upper function 
for local modulus of continuity of xe on G := Bd(t, r). The simplest consequence of this result will 
be the law of iterated logarithm (LIL) for \x0 — Xt| as well as its non-asymptotical version. Studying 
the local modulus of continuity we are obviously interested in the case when r is small even r — )■ 0. 
Thus, without loss of generality we will assume that r < 1. 

To apply Proposition 4 we need to define the function qa, A{-) = V2p{- — t), to choose the 
function si E S^fi, and to compute the function £'(^)(A, t), A G (0, r], e E (O, \/2 — l] , given by 

f(^)(A, t) = eif (5a([1 + e]A) ,Bd(t, [1 + e]A)) , a = V2p. 
Introduce the function 

iP{x) = V2 sup p{ei,e2), xe{o,2], 

ei,6'2eBd (t,l): 

d[ei,e2)<x 

and suppose that ip(2) < oo. Note that obviously ^(0) = 0, since d is a metric, and is increasing. 
Moreover, for any u G (0, r] 

g\{u):=V2 sup p(0, t) < V(^x), 
e-. d(e,i)<u 

that allows us to put qa = V'- Denoting the inverse function of we assert that Vu G (0, r] 

e^B,(t,«),v^pW<e^B,(t,n),d(V'-'(^)) <ln(Ard)([log2{VV'-'(^)}]+ + l), V5>0. (3.1) 

Hence, if the function ip is such that Assumption 3 is fulfilled then Proposition 4 is applicable and 
that provides us with the upper function for \xe — Xt| on Bd(t, r). However, this upper function 
does not admit an explicit expression, in particular its dependence on the variable A cannot be 
analyzed in general. So, we prefer to impose an additional assumption on the function tp that allows 
us to obtain the explicit expression of the upper function for \xe — Xi\ ^iid analyze it as well as the 
corresponding probability bound for small values of the radius r. We will not be tending here to 
the maximal generality and suppose that there exist < c < c < oo and /3 > such that 

cu^ < tlj{u) < cu^, Vu G (0, 1]. (3.2) 
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For example, if d = p one has = V^'u, and, therefore, (3.2) holds. Under (3.2) obviously 

{5/cy < i^-'^id) < and we get from (3.1) 



^B,(t,.),v^pW<ln(iVd) log2{n 



(5/c)"^} 



+ 1 , V5 > 0. 



(3.3) 



Taking into account that gA{u) = ip{u) > cu^ and choosing s{x) = s*{x) = (6/7r^)(l + [Inx]^) ^, 
we obtain from (3.3) for any A € (0, r] 



^(^)(A,t) = sup<^-^^^(^^j^^^j^)^^^(,4[l + e]A)(485)-.('^)) 
< ln(iVd)sup5"2 (/3^i 



5>0 



log2 i — lofe2 



s*((5) 



+ 1 =:C(/3,c,c,d). 



As we see £'^^-'(A,t) is independent of A,t and bounded from above by the constant which is 
completely determined by the triplet (T, d, x) and by the quantities c, c and /3. Thus, in view of 
Proposition 4 the upper function for has the following form (see also Remark 5). 



i>i''^)(A,t) = c(l+e)i+2/3A^./2[l + e-i]'C(/3,c,c,d) + (l + e)2 £(2r/A) + 



where, remind, i(y) = In {1 + In (y)} + 2 In {1 + In {1 + In (y)}}, y > 0. 

Choose z = z{r) = In |l + In |l + | In (r) | }} and e = e(r) := z~^{r) and define 



a(r) = (l + e(r)) 



p(r) 



1+2/3 



sup A 

A6(0,r] \ 



2[l + e-i(r)]^C(/3,c,c,d) + (!+£( 


r))2 


i 


[2r/A] 


+ z{r) 


ln{l + In (A) 


} 



2 + 


ln{l + ln(l + e(r))} 


-2 


l + ln{l+ ln(r) } 



We note that if r — )• then 

'ln{l + ln{l+ |ln(r)|}}" 



a(r) = 1 + 



In { 1 + I In (r) I } 



p(r) = O 



[ln{l +ln{l + 


ln( 


r) 




ln{l + 


In 


ir)\} 





(3.4) 



The following result is immediate consequence of Proposition 4. Put m(A) = sup^g^^^^) \xe — Xi\- 

Theorem 2. Let % he doubling space and suppose that (3.2) holds. Then, we have for any t G T 
and any r G (0, 1) 



P < 



sup 

Ae(o,rl 



m(A) 



cA/5Jln{l + |ln(A)|} 



> o(r) ) < 8p(r). 



The first consequence of Theorem 2 is the law of iterated logarithm. Indeed, taking into account 
that p(r) —7- 0, o(r) — t- 1, r — t- 0, we come to the following assertion. 
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Corollary 2. Let T be doubling space and suppose that (3.2) holds. Then for any t G T P — a.s. 



lim sup 



m(A) 



A/3 Jin {1 + I In (A) I } 



< c. 



We note, that although the statement of Corollary 2 is traditional in probability theory, the 
non-asymptotical statement of Theorem 2 is much more informative. 

The next consequence of Theorem 2 seems more curious. We remark that if T is doubling with 
respect to the intrinsic semi-metric p, then the normalizing sequence appeared in the theorem is 
independent of p. Moreover, the function a(-) depends only on p. 

Indeed, \i d = p then (/>(n) = \/2'u, and therefore, /3 = 1 and c = c = \/2- It yields, in particular, 
that C(/3,c, c, p) = CATp, where 

Civ, = ln(iVp)supr2 ([3 + 21og2 (vr) + loga {<5(l + [In }] + l) , 

<5>0 ^ ^ 

and, therefore, a(-) = ciArp(-), where aNp{-) is completely determined by p via the quantity Np. 
Corollary 3. Let % be doubling space with respect to d = p. Then, for any t G T and r G (0, 1) 



sup 

Aefo.rl 



m(A) 



AJ21n{l + |ln(A)|} 



> aNAr) } < 8p(r). 



We note that if W be the set of metrics p for which T is doubling and such that Np < N* for 
some fixed A^* G N* then the function aNp{-) in the assertion of Corollary 3 can be replaced by the 
universal on function OAr*(-). The corresponding inequality becomes "metric free". 

We finish this section by the consideration of several examples, where the condition (3.2) is 
verified. In these examples T is either M*^ or ]R^|_,ci > 1, x is Lebesgue measure and d is the 
euclidian distance. 



Example 1. Levy function Here xtj t £ I^'^i is zero-mean gaussian random field with p = \fd. 
Hence, (3.2) holds with c = c = y/2 and (5 = 1/2. We deduce from Theorem 2 that 




m(A) 



2Aln{l + |ln(A)|} 



> o(r) ) < 8p(r), r G (0,1). 



Example 2. Fractional brownian motion Here Xi-, t ^ is zero-mean gaussian random 
process with p = d"/^, a G (0, 2]. Hence, (3.2) holds with c = c = and /3 = a/2. We get from 
Theorem 2 




m(A) 



2A° ln{l + I ln(A)|} 



> a(r) ) < 8p(r), r G (0,1). 
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Example 3. Ornstein-Uhlenbeck process Here xt, t S M+, is given by 



Xt 



2A 



where h is the standard Wiener process. In this case p = A-i/^v/l -exp{-A4. Since we consider 
r G (0, 1) then (3.2) holds with c = cr^/2e~ie-^/^ c = /3 = 1/2 and we have 



P ^ sup 

Aefo.rl 



m(A) 



crJ2Aln{l + I ln(A)|} 



> a(r) }- < 8p(r), r G (0,1) 



4. Generalized empirical processes 

Let (Af,X, z/) be cr-finite space and let (r2,2l, P) be a probability space. Let Xj, z > 1, be a the 
collection of ^Y-valued independent random variables defined on (17,21, P) and having the densities 
fi with respect to measure v. Furthermore, Pf, f = (/i,/2,...), denotes the probability law of 
{Xi,X2, . . .) and Ef is mathematical expectation with respect to Pf. 

Let G:^xA'— T-Mbea given mapping, where ^ is a set. Put Vn G N* 

n 

Ch(n)=n-i^ [g((i,X,) -EfG(f),X,)], f)Gi3. (4.1) 

i=l 

We will say that i^{n), f) G is generalized empirical process. Note that if [) : A" — )• M and 
G{\],x) = \]{x), f) G i3,x G Af, then £,^{n) is the standard empirical process parameterized by Sj. 
Throughout this section we will suppose that 

Goo(f)) := sup |G(f),x)| < oo, Vt) G ij, (4.2) 

and it will be referred to hounded case. Some generalizations concerning the situations where this 
assumption fails are discussed in Section 4.1. 

The condition (4.2) implies that the random variables G(f), Xj), f) G ^, and G(f)i, Xj) — G({}2, Xj), 
f)i5 f)2 £ -Q, ^ = li '^j are bounded, and we obtain in view of Bernstein inequality Vz > 



^i{\ii,{ri)\> z) < 2exp|- 



^Ak^M-i^A^) >4 < 2exp j- I. , J . (^-4) 

l-l J I a^(f)i,f)2) + 2boo(lli,f)2) J 

where 

n n 

^2(fj) = 2n"2^E^G2(f),X,), a2(f,i,f,2) = 2n-2^E^(G(f)i,X,)-G(f)2,X,)ijl;5) 

i=l i=l 

5oo(W = (4/3)r^-'sup,g;t|G(f),x)|, boo(f)i,f)2) = (4/3)n-isup|G(f)i,x)-G(f)2,x)(4.6) 

We conclude that Assumption 1 is fulfilled with ^{■) = | • |, A = A{, B = Boo, a = af, b = boo 
and c = 2. 
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It is easily seen that af and boo are semi- metrics on Sj. We note also that ^, : — )• M is P-a.s 
continuous in the topology generated by boo- Thus, if C is totally bounded with respect to 
a/ V boo and such that A{ := sup(,g^ (f)) < oo, Boo ■= sup(,gy^ -Boo(f)) < oo, then we conclude that 
Assumption 2 is verified. 

Thus, in the problems for which Assumption 3 is verified the machinery developed in Propositions 
2-3 can be applied for f) G i^. We would like to emphasize, however, that problems studied 

below are not always related to the consideration of G with Sj being totally bounded, 

although such problems are also studied. The idea is to reduce them (if necessary) to whose for 
which one of Propositions 2-3 can be used. For instance, we will be interested in finding upper 
functions for on t) ^ Sj not only for given n but mostly on'N x Sj, where N is a given subset 

of N*. It will allow, in particular, to study generalized empirical processes with random number of 
summands. 

However the application of Propositions 2-3 requires to compute the functions £ or £ and there 
is no a general receipt how to do it. The main goal of this section is to provide with rather general 
assumptions under which the latter quantities can be computed explicitly. As it was already men- 
tioned in Introduction upper functions for random objects appear in various areas of probability 
theory and mathematical statistics. As the consequence the different nature of problems requires 
to specify the imposed assumptions. The assumptions presented below are oriented mostly to the 
problems arisen in mathematical statistics that definitely reflects author's scientific interests. How- 
ever, some pure probabilistic results like the law of iterated logarithm and the law of logarithm will 
be established as well. 

4.1. Problem formulation and examples. Main condition 

In this section we find upper functions for several functionals of the generalized empirical process 
defined in (4.1) under condition (4.2). We remark that the parameter t) may possess a compos- 
ite structure and its components may have very different nature. In order to treat such situations 
it will be convenient for us to assume that for some m > 1 

Sj=S)iX---xS)„i, (4.7) 

where Sjj, j = l,m, be given sets. We will use the following notations. For any given k = 0,m put 

Sji = f^i X ■ ■ ■ X f:)k, = S)k+1 X ■■■ X Sjm, 

with the agreement that ^5 = ^5 ^m+i = ^- "^^^ elements of Sji and -^^i will be denoted by t)^''"' 
and respectively. We will suppose that for any j = k + l,m the set Sjj is endowed with the 
semi- metric qj and the Borel measure Xj. 

In the next two sections we find upper functions for |^(,(n)| on on some subsets of (possibly 
depending on n\) and we will consider two cases. 

Totally bounded case. In this case we will suppose that S^j is totally bounded with respect to qj 
for any j = k + l,m. 

Partially totally bounded case. Here we first suppose that for some p > 1 

{X, v) = [Xi X ■ ■ ■ X Xp, z^i X • • • X fp) , (4.8) 

where {Xi, vi) / = l,p, are of measurable spaces and v is the product measure. 
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Next we will assume that f)m = Xi. As the consequence, the assumption, that S)rn is totally 
bounded, is too restrictive. In particular, it does not verified in the case X = Xi = which 
appears in many examples. Before to start with the presentation of the results let us consider 
several examples. 

Example 1. Density model. Let ii' : M'^ — )• M be a given function compactly supported on 
[-1/2, 1/2Y and denote for any /i = (/ii, . . . , hd) G (0, 1]'' 

-1 



K{-/h,,...,-/hd), 

J=i ' 

where, as previously, for two vectors u,v ^ the notation u/v denotes the coordinate- vice division. 

Put p = 1, m = d + 1, k = d, Xi = S)d+i = = (0,1], i = l,d and consider for any 

i] = {h,x)eSj := (0, if X R'^ 



^i){n) = ih,x{n) := n 



i=l 



Kh{Xi-x)-¥.,{Kh{Xi-x)} 



We have come to the well-known in nonparametric statistics kernel density estimation process. Here 
the function if is a kernel and the vector h \s a multi-bandwidth. 

Example 2. Regression model. Let ei,i = l,n, be independent real random variables dis- 
tributed on X C M and such that Ee, = for any i = l,n. Let Yi,i = l,n, be independent 
d-dimension random vectors. The sequences {ej,i = l,ra} and = 1,^} are assumed indepen- 

dent. Let be a given set of d x d invertible matrices and let X C M and Xi C M*^ be given 
interval. 

Fut p = 2, m = d+2, k = d, Xi = = M.'^, X2 = X, Sjj = (0, 1], j = T~d and Sjd+i = M. 

Consider for any = {x, h,M) eSj := (0, l]'^ x x R'^ 



IT' 

ei,(n) = ^h,MA^) ■= n-^\det{M)\Y,Kh[M{Y, - x) 



The family of random fields ^S,x,h,M{n), x,h,M G (0,1]^^ x A4 x appears in non-parametric 

regression under single index hypothesis, Stone (1985). 

If X is bounded interval, i.e. are bounded random variables, then (4.5) and (4.6) hold and the 
results from Section 2 are applicable. However this assumption is too restrictive and it does not 
satisfied even in the classical gaussian regression. At the first glance it is seemed that if X = M 
Propositions 2-3 are not applicable here. Although the aforementioned problem lies beyond of the 
scope of the paper, let us briefly discuss how to reduce it to the problem in which the machinery 
developed in Propositions 2-3 can be applied. 

Some generalizations. Let [ei,i = l,n) be the sequence of independent real-valued random 
variables such that Ee^ = (later on for simplicity we assume that Si has symmetric distribution) 
and Ee? =: af < 00. Let Xi, z = 1, n, be a the collection of valued independent random elements 
and suppose also that = l,n) and (ej,i = l,n) are independent. Consider the generalized 

empirical process 

n 

e(,(n) = n-iJ^G(f),X,)ei, i)eSj, 

i=l 
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where, as previously, G : Sj x X ^ M he a, given mapping satisfying (4.2). For any y > define 

n 

^i)i'^,y) = 'n~^^G{i),Xi)eili_y^y^{ei), r]n{y) = sup_|ei|[l - l[-y,y](ei)] ■ 

i=l *=l,n 

Obviously, for any y > 

n 

1=1 

where Gy{l),x) = (u), x = (x,u) £ X := X x M., t) £ Sj. Since Gy is bounded for any 

y > the inequalities (4.3) and (4.4) hold and, analogously to (4.5) and (4.5), we have 

n n 

A^{t)) = 2n-^Y.afEfG\\:),Xi), alit)u^2) = 2^-2 ^ a2E/(G([ii, X^) - G(f)2,Xi))'; 

i=l 1=1 

Soo(f)) = (4y/3)n-isup,e;t|G(f),x)|, boo(f}i,f)2) = (4/3)yn-i sup |G(f)i,x) - G(f)2,x)|. 

Let also C be such that the results obtained in Propositions 2-3 are applicable to |Cfi(^5y)| on 
for any y > 0. It is extremely important to emphasize that neither Af{-) nor af (•, •) depend on y. 
This yields, in view of Theorems 3 and 5 below, that upper functions for |^fi(y)| , i) £ Sj (for 

brevity V{i},y) and Uq{\)^y),q > 1) can be found in the form: 

V{i),y) = ViH))+yV2{i)), f/,(f),y) = [/g,i(f)) + y[/,,2(f)). 
It means that we are able to bound from above any y > 



q 



sup [Ct,{n,y) y] >oL Ef \ snp [Ct,{n,y) - Uq{i),y] 



Moreover, we obviously have for any y > 



sup[ef,(n)-F(f),y)] >0^ <P/<^ sup [^i,{n,y) - V{i),y] > U {7?„(y) > 0} ; 
EJsup[4(n)-[/g(f),y] V < E J sup [e„(n, y) - C/,(fi, y] V + f supGoo(f))yE (7?„)'' . 

Typically, V{-,y) = V^^\-,y) and [/,(•, y) = t/i"^(-,y) and vi''\-) « f/") and C/g(-) « C/^^i^ for 
all n large enough. It allows to choose y = yn m optimal way, i.e. to balance both terms in latter 
inequalities, that usually leads to sharp upper functions V^i"^(-) + ynV^2''"'*(") ^^"^ ^qi (') ~^VnU'^2i')- 

Main Assumption Now let us come back to the consideration of generalized empirical processes 
obeying (4.2). Assumption 4 below is the main tool allowing us to compute explicitly upper functions. 
Introduce the following notation: for any 

Goo(f)^''^)= sup sup|G(f),x)|, 
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and let Goo : -^i — ^ ^+ be any mapping satisfying 

Goo(fl(')) <Goo(fl('0, Vf,« Gi^t. (4.9) 

Let {Sjj{n) C Sjj, n > 1} ,j = l,k, he a. sequence of sets and denote io^(n) = i3i(n) x • • ■ Sjk{n). 
Set for any n > 1 

G„= inf Goo(f)('=)), G„= sup Goo(fl('=^). 

For any n > 1, j = 1, /c and any f)j € S^j{n) define 
Gj,n%)= sup Goo(f)('^)), G^-„= inf G,,oo(f)i). 

hi6fii{n),...,l)j_i6^lj_i(n),()j + iG%+i(n),...,f)fe6iOfe{n) hje%(n) 

Noting that | In (ti) — In (t2) | is a metric on R_)_ \{0}, we equip Sji{n) with the following semi- metric. 
For any n > 1 and any i)^''^ G ^i{n) set 

= ln{G,>(6,)} -ln{G,-„(^j)} 

where ^j, j = 1, /c, are the coordinates of [}^^^ and t}^^^ respectively. 
Assumption 4. (i) < G„ < G„ < oo /or an?/ n > 1 and for any j = l,k 

> Vf)(^) = (b, . . . , f).) e ^Hn), Vn > 1; 

(ii) There exist functions Lj : ]R+ — t- ]R+, Dj : M+ — )• M+, j = 0,k + 1, . . . ,m, satisfying Lj 
non- decreasing and bounded on each bounded interval, Dj G C^(M), D{0) = 0, and such that 



|G((,,.)-G(^,.)L < {Goo(f)(^'))vGoo(^('=^)}z)o{^)(f)('=\^^'^)} 

m 

E L,{Goo(fl('=)) VGoo(^('-))}z?,(^,,(f,,,f,;.)), 



+ 

for any f), f)' G i^i(n) x Sj^_^^ and n> 1. 
We remark that Assumption 4 (i) is automatically fulfilled if A; = 1. 

Remark 6. If n > I is fixed or S)j{n), j = l,k, are independent on n, for example S)j{n) = 
S^j, j = l,k, for all n > 1 then upper functions for \^h{''^)\ can be derived under Assumption 4- 
However, if we are interested in finding of upper functions for \£,hin)\ when n is varying, we cannot 
do it in general without specifying the dependence of S)j{n), j = l,k, on n. 

In view of latter remark we will seek upper functions for when f) G Sj{n) := S)i\{n) x Sy^j^^. 

Here S^\{n) = S^i{n) x ■ ■ ■ S^k[n) and |i^j(n) C S)j{n), n > l| ,j = 1, fc, be a sequence of sets 
satisfying additional restriction. We will not be tending here to the maximal generality and complete 
Assumption 4 by the following condition. 

Assumption 5. For any m G N* there exists n[m] G {m, m + 1, . . . , 2m} such that 

U ^f(n)Ci5^(n[m]). 

nG{m,m+l,...,2m} 

We note that Assumption 5 obviously holds if for any j = l,k the sequence n > l| is 

increasing/decreasing sequence of sets. 
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4 •2. Totally hounded case 



The objective is to find upper functions for |^(j(n)| under Assumption 4 enforced, if necessary, by 
Assumption 5 and the condition imposed on the entropies of the sets j = k + l,m. 

4-2.1. Assumptions and main result 

The following condition will be additionally imposed in this section. 

Assumption 6. Suppose that (4-7) holds and there exist N,R < oo such that for any ? > and 

any j = k + l,m 

ei-,,,,,(?)<A^([log2{i?A}]+ + l), 

where, as previously, ^f)j,Qj denotes the entropy of f)j measured in qj. 

We remark that Assumption 6 is fulfilled, in particular, when (^S)j, gj, , j = k + l,m, are 
bounded and satisfy doubling condition. Note also that this assumption can be considerably weak- 
ened, see discussion after Theorem 3. 



Notations Let 3 < ni < n2 < 2ni be fixed and set N = {ni, . . . , n2}. For any i) €z Sj set 

' sup-^Y^E{\G{\:},Xi)\, ni/n2; 
, (n2)-iEr=iIEf|G(t),X,)|, ni=n2. 



^n.(fl) 



and remark that if additionally Xi, i > 1, are identically distributed then we have the same 
definition of i^n2(") i^i both cases. We note that 

:= sup sup Fn2(f)) < sup G„ < oo 

in view of Assumption 4 (i). Let 5 > 1 be fixed and put 

^^fni, ni = n2; B = i = n2; 

\ n[ni], ni/n2, \ b, ni / n2, 

where, remind, that n[-] is defined in Assumption 5. 

Define Lj(z) = max {z-^Lj{z), 1} and = EJLfc+ilog2 {Lj (2^)} and introduce the fol- 

lowing quantities: for any [}^^^ G S^^ and any q > 

p(f)W) = {3Qk6:^+6)ln(^l + ln[2G:^^Goo{i)^^'^)})+S6N6:^£^''^(^Go,{^^^^^^ 



Here (5* it is the smallest solution of the equation (48(5) ^s*((5) = 1, where, remind, s*{6) = 
(6/7r2)(l + [In 6]^y , 6>0. The quantities N,R are defined in Assumption 6. 

The explicit expression of the constant G]y^ji^rn,k7 as well as explicit expressions of the constants 
Ai, A2 and Cd^i, used in the description of the results below, are given in Section 7.1.2 which 
precedes the proof of Theorem 3. 
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Result For any r G N put -Fn^.rlf)) = max [F^^ (f)), e and define for any £ Sj, u > and q > 
Vi")(n,f)) = Ai./Goo(f)W) (F„„r(f))n-i) (p(f,W) + 21n{l + |ln (F„„r(f)))|} + 



+A2Goo(f)(')) (n^' In'^ (n)) (p([)(^)) + 21n{l + |ln (F„„r(f)))|} + 



Ut'\n,[)) = Ai./Goo(f)(^))(F„„r(f))n-i)(M,(t)('=))+21n{l + |ln(F„,,r(W)|} + 



+A2Goo(fl('))(n^Mn'5 (7^)) (Af,(fi('=)) + 2 In {1 + |ln (F„,,r([)))|} + 

Theorem 3. Let Assumptions 4 and 6 be fulfilled. Ifn^ ^112 suppose additionally that Assumption 
5 holds. Then for any r£N,b>lu>l and q > 1 



Ef < sup sup 

[neNfi6^(n) 



"^f < sup sup 



ef,(n)| - V^^(n, f))l > I < 2419 e""; 



< Cn 



V(ni)-iFn,G„ V (ni)-i In/^ (n2)G, 



w/iere Cq = 2(79/2)+53g+4r(q + l)(Cz),6)''. 

Remark 7. T/ie inspection of the proof of the theorem allows us to assert that Assumption 6 can he 
weakened. The condition that is needed in view of the used technique: for some a G (0, 1), L < 00 



sup? < -L, j = k + l,m. 

f>0 



(4.10) 



In particular, it allows to consider the generalized empirical processes indexed by the sets of smooth 
functions. However the latter assumption does not permit to express upper functions explicitly as it 
is done in Theorem 3. This explains why we prefer to state our results under Assumption 6. 

Several other remarks are in order. 

1°. First we note that the results presented in the theorem are obtained without any assumption 
imposed on the densities /j, i > 1. In particular, found upper functions remain finite even if the 
densities fi, i>l are unbounded. 

2". Next, putting r = +00 we get the results of the theorem with i^n2,r(') = Fn-2{')- It improves 
the first terms in the expressions of Vr"^(-, •) and Ur^''^\-, •), however the second terms may explode 
if Fn^i^) = for some t) £ Sj. The latter fact explains the necessity to "truncate" F,^^{-) from 
below, i.e. to consider Fn2,r(') instead of F^^i'). 



4-2.2. Law of iterated logarithm 

Our goal here is to use the first assertion of Theorem 3 in order to establish a non-asymptotical 
version of the law of iterated logarithm for 

r?t,(fc)(n):= sup |^fi(^)|- 



Let us suppose that for some c > 0, b > 



c<G„<G„<cn^ Vn>l. 
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(4.11) 



We would like to emphasize that the restriction > c is imposed for the simplicity of the notations 
and the results presented below are valid if G„ decreases to zero polynomially in n. 
Moreover we will assume that 



sup sup supEf =: F < oo. 
n>l f,efl(n) *>i 



(4.12) 



We will see that the latter condition is checked in various particular problems if the densities 
fi, i > 1 are uniformly bounded. Suppose finally that for some a > 



< oln{l + ln(z)}, Vz > 3. 



(4.13) 



For any a > and n > 3 define 



i3?(n,a)=^f(n)n{f)('=) : Goo (fl^'^^)) < n[ln(n)] . 

Theorem 4. Let Assumptions 4, 5 and 6 he fulfilled and suppose additionally that (4-11), 
and (4-13) hold. Then there exists T > such that for any j > 3 and any a > 2 



Pf < 



sup 



sup 



_n>j L^Goo(f)(^'))ln(l + ln(n))J 



> T > < 



2419 
Ml)' 



The explicit expression of the constant T can be easily derived but it is quite cumbersome and 
we omit its derivation. 

Remark 8. The inspection of the proof of the theorem shows that for any y > one can find 
< T(y) < oo such that the assertion of the theorem remains true if one replaces T by T(y) 



-(i+y) 



and the right hand side of the obtained inequality by 2419 [ln(j) 
consideration of small values o/j. 

The simple corollary of Theorem 4 is the law of iterated logarithm: 



It makes reasonable the 



lim sup sup 



Goo(fl(^'))lnln (n) 



< T, 



- a.s. 



(4.14) 



4.3. Partially totally bounded case 

We begin this section with the following definition used in the sequel. Let T be a set equipped with 
a semi-metric d and let n G N* be fixed. 

Definition 1. We say that {T; C T, i € 1} zs n-totally bounded cover ofT if 

• T = UigiT; and I is countable; 

• Ti is totally bounded for any i G I; 

• card( {k G I : Ti n Tk / 0} )< n /or any i G I. 
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Let us illustrate the above definition by some examples. 

Let T = W^, d> 1. Then any countable partition of consisted of bounded sets forms 1-totally 
bounded cover of M'^. Note, however, that the partitions will not be suitable choice for particular 
problems studied later. We will be mostly interested in n-totally bounded covers satisfying the 
following separation property: there exists r > such that for all i, k G I satisfying Ti n = 

inf d(ti,t2)>t. (4.15) 

Let us return to that we equip with the metric generated by the supremum norm. Denote by 
Br(i), t G W^, r > 0, the closed ball in this metric with the radius r and the center t. For given r > 
consider the collection |]Br(ri), i e Z'^j, where we understand ri as coordinate-wise multiplication. 

It is easy to check that this collection is S'^-totally bounded cover of M'^ satisfying (4.15). 

We would like to emphasize that n-totally bounded covers satisfying the separation property 
can be often constructed when T is a homogenous metric space endowed with the Borel measure 
obeying doubling condition. Some useful results for this construction can be found in the recent 
paper Coulhon et al. (2011), where such spaces were scrutinized. 

We finish the discussion about n-totally bounded covers with the following notation: for any 
t G T put 

nt)= u u 

iel: tell kel: TinTk7^0 



4-3.1. Assumptions and main result 

Throughout this section we will assume that the representation (4.8) holds and the elements of 
Xi, I = l,p, will be denoted by x; . We keep all notations from previous section and replace 
Assumption 6 by the following conditions. 

Assumption 7. (i) Let (4-7) and (4-8) hold with Xi = Sjm and for some n G N* there exists a 
collection |Hm,i, ^ ^ -'-| ^^zng the n-totally bounded cover ofSjm satisfying for some N,R < oo 

eH^„,™(?) < ([iog2 + 1) , vi G I, > 0. 

(ii) For any <j > 

G:i-,„e. (?) < ^^ ( [log2 {R/^}] + + l) , Vj = A; + l,m-l. 

Usually one can construct many n-totally bounded covers satisfying Assumption 7 (i). The con- 
dition below restricts this choice and relates it to properties of the mapping G{-,-) describing 
generalized empirical process. 

Assumption 8. For any n> 1 and any t) = (f)i, . . . , t)m) G S^{n) 

sup |G(f),x)| <n^iGoo(t)(')). 

We would like to emphasize that in order to satisfy Assumption 8 in particular examples, the 
n-totally bounded cover |Hm,i, ^ ^ -'-} should usually possess the separation property. Indeed, one 
of the typical examples, where Assumption 8 is fulfilled, is the following: there exist 7 > such 
that for G{x, f)) = for any x £ X, f) G i^, satisfying pm{xi, f)m) > 7- 
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Result For any i = 1, n we denote Xi = • • • , Xp^i^ , 



and \i X = Xi {p = 1) then we put Xi^i = Xi and fi^i = fi. 
Put for any n > 1, v > and any fj^ G -^m 



In 



n „ 

n"^^ / /i,i(x)z/i(dx) 



Note that obviously < S^n,v{hm) < vln{n), Vf)m G -^m- Put for any \j £ Sj 

= P(f)(^)) +£„,,,(f)™) +21n{l + |ln(Fn,,r(f)))|}; 
M,([)) = M, ( f) f)^) + 21n{l + |ln(F„,,r((l))|}. 

Define for any {)Gi3,rGN, z>0 and q > 

V^^) (n, {)) = Ai jGoo({)W)(F„„r(f})n-i)(P(fi)+z) + A2G00 (f)^'^)) (n^^ In^ (n)) + 2) ; 



n, 



[)) = Am/Goo (f)^'^)) (F„„r(Wn-M (M,([i) + + A2Goo(f)('=)) (n^' In^ W ) (M,(f)) + 



Theorem 5. Let Assumptions 4, 7 <ind 8 hold. If 7^ n2 suppose additionally that Assumption 
5 holds. Then for any rGN, f>l, z>l and q > 1 



Ff < sup sup 
Ef < sup sup 

[neN ()65(ii) 



|e„(n)|-V('''^)(n,t)) 



> )> < n^<j 4838e-^ + 2ni2-''}; 



V(ni)-iF„,G„ V (ni)-i In^ (n2)G„ 



+2''+in5(Gn)''ni 



Although the assertions of the theorem are true whenever f > 1 the presented results are obvi- 
ously reasonable only if u > 2. For example (as we will see later) the typical choice of this parameter 
for the "moment bound" is v = q + 2. 

In spite of the fact that upper functions presented in Theorem 5 are found explicitly their 
expressions are quite cumbersome. In particular, it is unclear how to compute the function 
Of course, since £,n,v{^m) < t'ln(n), Vf)m £ Sjrm one can replace it by t;ln(n) in the definition of 
P(-) and Mq(-), but the corresponding upper functions are not always sufficiently tight. 

Our goal now is to simplify the expressions for upper functions given in Theorem 5. Surprisingly, 
that if n is fixed, i.e. ni = n2, it can be done without any additional assumption. 

Set for any v > and f) € 

P,(fl(*=)) =P(t)('=))+27;|ln(2Goo(f)('=)))|, M,,.(fl(^)) = M,(f)W) + 2t; |ln (2G00 (f)^'))) 
and let Fn^ih) = max[Fn2 (t]), n2~^]. 
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Corollary 4. Let the assumptions of Theorem 5 hold. Ifni ^ n2 suppose additionally that Xi^i, i > 
1, are identically distributed. 

Then, the results of Theorem 5 remain valid if one replaces vi^'^\n,t)) and Ur^''^'''\n,t)) by 



^ Ai^ Goo(fl(^^)) (i^n.(f))n-ij (P.(il«) + 2{v + 1)1 In {^^.(f,)} | + z 
+A2G00 (n-i In" (n)) (p, (fi + 2{v + 1) | In (W } | + ; 



We would like to emphasize that we do not require that Xi, i > 1, would be identically dis- 
tributed. In particular, coming back to the generalized empirical process considered in Example 2, 
Section 4.1, where Xi = (li,ej), the design points Yi, i > 1, are often supposed to be uniformly 
distributed on some bounded domain of M.'^. As to the noise variables Sj, i > 1, the restriction that 
they are identically distributed cannot be justified in general. 



^ AiyGoo(fl(*^)) [P^Mn-') (M„.(f)('-)) + 2{v + 1)| In {Fn.(f))}| + 
+A2G00 (n-i In/' (n)) (M,,,(f)W) + 2{v + 1)| In {F^M}\ + 



4-3.2. Law of logarithm 

Our goal here is to use the first assertion of Corollary 4 in order to establish the result referred 
later to the law of logarithm. Namely we show that for some T > 



lim sup sup 



< T Pf - a.s. 



i,Wefli(n,a) JGoo(f)(^)) ln{Goo(f)('=))} Vlnln 



(4.16) 



As previously we will first provide with the non- asymptotical version of (4.16). 

We will suppose that (4.11) and (4.12) are fulfilled and replace (4.13) by the following assumption. 
For some a > 

£(^')(z) < aln(z), Vz > 2. (4.17) 

Theorem 6. Let Assumptions 4, 5, 7 and 8 be fulfilled. Suppose also that (4. 11 ), (4-1^) '^'^^ (4-1'^) 
hold and assume that Xi^i, i > 1, are identically distributed. 
Then there exits T > such that for any j > 3 and any a > 4 



sup sup 



> T > < 



iiWeii^(n,a) JGoo ln{Goo(f)W)} Vlnln 



n 



4840n^ 
1^ 



Some remarks are in order. The explicit expression of the constant T is available and the gener- 
alization , similar to one announced in Remark 8, is possible. Also, (4.16) is an obvious consequence 
of Theorem 6. At last, we note that in view of (4.11) the factor In |Goo (J)^*'^) } V Inln(n) can 

be replaced by ln(n) which is, up to a constant, its upper estimate. The corresponding result is, of 
course, rougher than one presented in the theorem, but its derivation does not require Xi^i, i > 1, 
to be identically distributed. This result is deduced directly from Theorem 5. Its proof is almost 
the same as the proof of Theorem 6 and based on the trivial bound -Cn,i)(f)m) ^ f In (n), Vt)m G ^m,' 
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4. 4- Application to localized processes 

Let (X;, /X;, Pi), I = l,d + 1, d G N, be the collection of measurable metric spaces. Throughout this 
section we will suppose that (4.8) holds with p = 2, 

X = XiX X2, [Xi,ui) = (Xi X ■■■ X Xrf,^i X ••• X ^rf) =: (Xf,/^^), (Afa, 1^2) = (X^+i, /i^+i) , 

Xj denotes the element of Xj, j = 1, and x^'^'> will denotes the element of X^ We equip the 

space X^ with the semi- metric p'^'^^ = max^^j-^ pi . 

Problem formulation This section is devoted to the application of Theorems 3 and 5 in the 
following case: 

. f)f := iOi X • • • X iid = (0, 1] X • • • X (0, 1] = (0, 1]^ (i.e. k = d); 

• ^d+i ~ ^d+i X := Z xXf , i.e. m = d + 2, where Xf := Xi x • • • x X^ be a given subset 
of X^ and is a given metric space. 

• The function G{-,-) obeys some structural assumption described below and for any f) := 
(r,3,xW) G (0,1]'^ X Z X Xf the function G(f),-) "decrease rapidly " outside of the set 

jxi G Xi : pi(xi,xi) < rij X • • • X jxrf G Xrf : pd{xd, xa) < r^^ x X^+i- 

Let : M"^ — )• M be a given function, (71, . . . , 7^) G M^J. be given vector and set for any r G (0, 1]*^ 

d 

1=1 

where, as previously, for u,v (zM.'^ the notation u/v denotes the coordinate-wise division. Let 

G(f), x) = 5(3, x)Kr (p(x('^), , d = (r, 3, G (0, l]'^ x Z x Xf =: ^, (4.18) 

where g : Z x X is a given function whose properties will be described later and 

p{x'^'''\x^'^^) = (pi(xi,xi), . . .,Pd{xd,Xd)) . 
The corresponding generalized empirical process is given by 

n r 

i^{n) = 9{hX,)Kr (/?( [X,]^ - [g{^,X,)Kr [p[ [X,]^ ,xW 

1=1 L 

We will seek upper functions for the random field Cr{''T', x^'^^) := sup j^^, ^ ^(t^)(?^)| in two cases: 

Xf = Xf and Xf = {x^} for a fixed x^'^) G Xf. 

To realize this program we will apply Theorems 3 and 5 to S,t){n), tj = ^r,3, x*-*^^^ . It is worth 
mentioning that corresponding upper functions can be used for constructing of estimation proce- 
dures in different areas of mathematical statistics: M-estimation with locally polynomial fitting 
(non-parametric regression), kernel density estimation and many others. 

Moreover, we apply Theorem 4 for establishing a non-asymptotical version of the law of iterated 
logarithm for (x*-"^^ , n) in the case where Xf = jx^'^-' } . We also apply Theorem 6 for deriving a non- 
asymptotical version of the law of logarithm for ||Cr(f^)|loo •= sup^{d)gx'' |Cr(a5'''^\ |- Our study here 
generalizes in several directions the existing results Einmahl and Mason (2000), Gine and Guillou 
(2002), Einmahl and Mason (2005), Dony et al. (2006), Dony et Einmahl (2009). 
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Assumptions and notations 

Assumption 9. (i) H-fi'lloo < oo and for some Li > 



\K{t)-Kis)\< 'I Vt,.GM^ 

1 + t A s 



where \ • | denotes supremum norm on W^. 

(ii) \\g\\oo '■= sup 5(3, x) < 00, and for some a € (0, 1], La > 0, 

sup \g{i,x) -g{}',x) \ < L„ [9(3,3')]" , Vjj' G Z; 

The conditions (i) and (ii) are quite standards. In particular (i) holds if K is compactly supported 
and lipschitz continuous. If 5(3, •) = g{-), for any 3 G 2^, then (ii) is verified for any bounded g. 

Let < r|-™"^''(n ) < r{°'^^^(n) < 1, / = n > 1, be given decreasing sequences and let 

d 

S3{n) = n{n) X Z X Xf, n{n) = J] [r'f^''\2n),r^^'"'\n)\; 

1=1 
d 

i^(n) = n{n) X Z X Xf, Ti{n) = J] [rf"^'°V), rf'^''"^(n)] . 

1=1 

We note that S){n) C S^[n) for any n > 1 since r\™^"\-),r\"^'^^\-), I = l,d, are decreasing, and 
obviously Sj{n) C i^(m) for any n G {m, . . . , 2m} and any m > 1. 

Remark 9. Assumption 5 is fulfilled with n[m] = m. 

Lemma 2. Suppose that Assumption 9 is fulfilled and let C he an arbitrary subset. Then, 
for arbitrary sequences < r^^™™^(n) < ri^^^\n) < 1, I = l,d, n > 1, Assumption 4 holds with 

Qn^ {r,r') = max 1 7; In (r;/r[) I, g^+i = [d]", Qd+2 = maxp;; 

l=l,d l=l,d 

Do{z) = ex.p{dz} - 1 + (Li/||/f||oo)( exp {j'^z} - 1 ), 7 = min7;; 
Dd+i{z) = {La/\\g\\oo)z, Dd+2{z) = (Li/||K||oo)^ Ld+i{z) = z, Ld+2{z) = z^. 

Additionally, if^f consists of a single point x^'^^ G X^ then Ld+2 = 0. 

The proof of the lemma is postponed to Appendix. We remark that Qd+i is a semi-metric, since 
a G (0,1], and the semi-metric g'n'^ is independent on n. In view of latter remark all quantities 
involved in Assumption 4 are independent on the choice of r^^^'^\-),r^^^\-), I = 1, d,. We want to 
emphasize nevertheless that the assertion of the lemma is true for an arbitrary but a priory chosen 

^(min)(.^^^(max)(.^^ ^^^^ 

Thus, Lemma 2 guarantees the verification of the main assumption of Section 4.1, that makes 
possible the application of Theorems 3-6. Hence, we have to match the notations of these theorems 
to the notations used in the present section. 
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bmce k = d and S^'l = (0, 1]"^ we have ^^'^^ = r and, therefore, in view of Assumption 9 



Goo(?') := sup _ sup 

(3,5:(d))e2xXf xexf+i 

Gn ■■= inf Goo{r) = V~^^^^^ \\g\\^\\K\\oo, Vn > 1. 
We remark that the function Goo(-) is independent of the choice of Xf. Define 

i > 1, 



and let 3 < ni < n2 < 2ni be fixed. Set for any (r,x('^)) G (0, l]'^ x Xf 



l^lloo sup 



and note that in view of Assumption 9 (ii) 



))|/('^)(xW)^W(dxW), 



ni = n2; 
ni / n2, 



Fn.(f)) < i^n. , Vf) G (0, 1]'^ X ^ X Xf. 

We remark that the function Fn^{-, •) is independent of the choice of Z. Put also 



:= sup sup (r, rr*^) < 



oo. 



"6N (r,x<^)67^(n)xXf 

where, remind, N = {ni, . . . , n2}. Finally for any r G N set Fn2,r(-, •) = max [F^^ (•, •), e"""]. 
4-4- 1- Pointwise results 

Here we will consider the case, where Xf = jx^*^)} and x^'^'^ is a fixed element of X^. Note that in 
view of Lemma 2 Ld+i{z) = z and ^^+2 = that implies C'^^^ = 0. 

We will suppose that Assumption 6 holds with k = d,m = d + 1 and {S^d+i, Qd+i) = (-2^) ['']")• 
It is equivalent obviously to assume that Assumption 6 holds with {Sjd+i, Qd+i) = {^^^) ™d with 
the constants N = aN and R = i?^/". 

Let /3 and C]\f^ji^jn,k be the constants defined in Theorem 3. Set for any r G (0, l]'^ and q > 



P{r) = (36(i(5-2 + 6) In 1 + ^ 7/ In <^ — ^ 



1=1 



Mg{r) = (72dC' + 2.5(7 + 1-5) 



1=1 



n 



+ i^CN,R,d+i,d; 

+ 36CAr,/?,d+l,d- 



30 



and define for r G N and n > 



Ai 
+A2 

Ai 

+A2 



(nVr)"'^ In^ (n) P(r) + 21n|l + 



'}\} 



u 



Fn^,r{r, x^) (nK-)-iJ [Mg{r) + 2 In { 1 + |ln {Fn,,r (r, x'^) } | } + u 
(nK)-i In^ (n)] ^Mg{r) + 2 In { 1 + lln |Fn,,r (r, x'^) } I } + 



where Ai = \/||ff||oo ||-f^||ooAi, A2 = H^Hoo ||-f^||ooA2 and Ai, A2 are defined in Theorem 3. 

The result below is the direct consequence of Theorem 3 and Lemma 2. We remark that defined 
above quantities are functions of r and n since x*^ is fixed. Since they do not depend on the variable 
3, these quantities will be automatically upper functions for 



Cr{n,x^'^^) := sup 



Theorem 7. Let Assumption 9 he fulfilled and suppose that Assumption 6 holds with k = d,m = 
d + 1 and {Sjd+i, Qd+i) = {Z, [O]"). 

Then for any given decreasing sequences < rf^''^\n) < r["'^''^(n) < 1, Z = 1, d, n > 1, any 



x"^ e Xf any rGN, 6>ln>l and q>l 



< sup sup 

[neNre'K.(n) 



Ef < sup sup 

[n6Nre7e(n) 



Cr(n,x('^)) -V^")(n,r,x'^) 
C.(n,xW) -a^"''')(n,r,x'^) 



< cL 



> 0| < 2419 e""; 



n-2 



V 



niK( 



/ = 2(7'?/2)+535+4r(g + 1) (CD,6max [^\\g\\^\\K\U, \\g\\oo\\K 



where c| 

The explicit expression for C/j 5 can be also found in Theorem 3. In the case considered here it 
is completely determined by (71, • • • 7d), Li, La and b. 

As well as the assertions of Theorem 3 the latter theorem is proved without any assumption 
imposed on the densities /j, i = \,n. The choice of rf^^'^\n),r'f^^^\n), I = l,d, n > 1, is also 
assumption free. Additionally, Assumption 6 can be replaced by (4.10), see Remark 7. 

Note also that if (7(3, •) = g{-), for any 3 G 2, then Assumption 6 in not needed anymore and, 
moreover. Assumption 9 (ii) is verified for an arbitrary bounded g. Hence, in this case the assertions 
of Theorem 7 are established under very mild Assumption 9 (i) imposed on the function K. 

Remark 10. We note that the discussed in Introduction so-called price to pay for uniformity 
disappears if r = r^^^^\ Indeed, p (^r^^^^^^^ and Mg (^r^^^^^) are absolute constants. This property 
is crucial, in particular, for constructing statistical procedures used in the estimation of functions 
possessing inhomogeneous smoothness, see Lepski et al. (1997), Kerkyacharian et al. (2001). 

Some additional assumptions and their consequences To apply Theorem 7 to specific 
problems one needs to find an efficient upper bound for the quantity Fn.^{-, •). Below we provide 
with sufficient condition allowing to solve this problem under general consideration and we will not 
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be tending here to the maximal generahty. We impose some additional restrictions on the densities 
fi, i = l,n, and on the measures /x; of p/-balls in the spaces X/, I = l,d. Moreover, we should precise 
the behavior of the function K at infinity. Then, we will use these assumptions for establishing of 
the law of iterated logarithm. 

Introduce the following notations. For any t E define 

K{t) = sup \K{u)\, Ut = [0,ti] X ••• X [0,td]. 

For any I = l,d, G X; and r > set B/(r,x/) = {y G X/ : pi(^y,xi) < r|. 
Assumption 10. There exists L2 > such that 



sup 



1=1 



< L2; (4.19) 



For any I = l,d and any xi € X/ one has X/ = Ur>o (Bi(r, x;)) and there exist L^'^ > 

^;(]B,(r,Xi)) < L(')rTS Vr > 0; (4.20) 

Moreover, 

sup sup //'^^(x^'^)) =: foo < 00. (4.21) 

i>l xW&Xf 

The condition (4.19) is obviously fulfilled if K is compactly supported on [0, 1]*^. It is also satisfied 
in the case of Gaussian or Laplace kernel. 

The condition (4.20) can be easily checked if X;, I = l,d are doubling metric spaces. In particular, 
if X; = M and ^u/, I = l,d, are the Lebesgue measures than (4.20) holds with L^''^ = 1, 7; = 1, I = 
l,d. If X; = E.'^' , I = l,d, then (4.20) holds with 7; = di and the constants L^'^ depending on the 
choice of the distances pi. 

As to condition (4.21) we remark that the boundedness of the entire density fi is not required. 
For example, under independence structure, i.e. fi{x) = fi'^\x^'^^)pi[xd+i), the densities pi may be 
unbounded. 

Lemma 3. The following bound holds under Assumption 10: 

d 

sup sup sup F^Jr^x^'^^) <2HMo.L2\{2^^L^^\ 



n2>lre(0,l] s(d)eXf 



1=1 



The proof of lemma is postponed to Appendix. Our goal now is to deduce the law of iterated 
logarithm for C,r{n^ x^'^^) from Theorem 4. Set for any n G N* and a > 

na{n) = {r G (0, 1]"^ : Vr > n~\ln?i)''}. 

and choose /i^"^^^) = (1, . . . , 1) and = (1/n, . . . , 1/n). 

Remark 11. 1°. Note that TZa{n) C [n^^l]'^ =: 7^(n) for any n > 3 and any a > and, 
therefore, the assertion of Lemma 2 holds. 

2". We have = \\K\\^\\g\\oo, Gn = ||-fC||oo lls'lloo''^"^ for any n > 1 and, therefore, (4-11) is 
verified with c = ||i^||oo Halloo (ind b = d. 
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3^. Lemma 3 implies that the condition (4-12) holds with F < 2'^foo||(7||oo-^2 rif=i 2'*^'-^^'^- 
4*^. In view of Lemma 2 Lfij^i{z) = z and ^^+2 = 0, that implies C^^^ = 0. Hence, the condition 
(4.13) is fulfilled for any a > 0. 

Thus, all assumptions of Theorem 4 are checked and we come to the following statement. 

Theorem 8. Let Assumptions 9 and 10 he fulfilled and suppose that Assumption 6 holds with 



k = d,m = d+1 and {Sjd+i, Qd+i) 
and any a > 2 



(Z, [5]°). Then there exists T > such that for any x G 



Pf ^ sup sup 

n>i r: n-^{\nn)'^<Vr<l 



Cr{n,x^'^'^) 
ln(l +ln (n)). 



> T > < 



2419 

W)' 



Remark 12. The inspection of the proof of Theorem 4 together with Lemma 3 allows us to assert 
that the statement of Theorem 8 is uniform over the set of bounded densities. 
More precisely, for any f > there exists T(f) such that 



sup Pf < sup sup 

l&Tf I ?i>j r: n-^{lnn)'^<Vr<l 



In (1 +ln (n)) 



> T(f) } < 



2419 

Ml)' 



(4.22) 



where J-j = {{fi,i > 1) : foo < f}- before the explicit expression o/T(-) is available. 
The following consequence of Theorem 8 is straightforward. 

'VnKCr{n,x^'^'^)' 



lim sup sup 

n-5>oo r: n-i(lnn)°<K-<l 



ln(l +ln (n)) 



< T 



a.s. 



(4.23) 



Theorem 8 generalizes the existing results, see for example Dony et Einmahl (2009), in the following 
directions. 

1. Structural assumption. The structural condition (4.18) is imposed in cited papers but with 
additional restriction: either g{]),x) = const ("density case") or g{i,x) = g{x) ("regression 
case"). It excludes, for instance, the problems appearing in robust estimation. We note that 
Assumption 9 (ii) is fulfilled here if g is bounded function and Assumption 6 is not needed 
anymore, since g is independent of 3. 

2. Anisotropy. All known to the author results treat the case where X; = M, I = l,d, and 
n{n) = {(n, . . . ,rrf) G (0, 1]"^ : = r, V/ = 17^, r G [r(™")(n),r(™'^^)(n)] } (isotropic case). 
We remark that (4.20) is automatically fulfilled with 7; = 1,L*^') = 1, I = l,d, and Vr = r'^. 
Note also that we consider independent but not necessarily identically distributed random 
variables. This is important, in particular, for various estimation problems arising in non- 
parametric regression model. 

3. Kernel. We do not suppose that the function K is compactly supported. For instance, one 
can use the gaussian or laplace kernel. It allows, for instaince, to consider the problems where 
Xf is not linear space. In particular, it can be some manifold satisfying doubling condition. 

4. N on- asymptotic nature. The existing results are presented as in (4.23). Note, however, that 
the random field Cr {n, x'^) appears in various areas of nonparametric estimation (density 
estimation, regression). As the consequence a.s. convergence has no much sense since there is 
no a unique probability measure (see, also Remark 12). 
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Sup-norm results 

Here we consider = . We assume that there exists {Xi, i G 1} which is n-totahy bounded 
cover of (^f,p^^^) satisfying Assumption 7 (i) and possessing the separation property. 

Assumption 11. There exists t > such that for any i, k G I satisfying Xi n Xk = 

inf inf p('^) (x^'^\y^'^A > i. 

Also we suppose that Assumption 7 (ii) holds with k = d,m = d+1 and (i^d+i, Qd+i) = {2, [f]"). 
We remark that in the considered case this assumption coincides with Assumption 6. 

Let, as previously, < r|'™™^(n) < r'^^^^\n) < 1, I = l,d, n > 1, be given decreasing sequences, 

d 

i3(n) = 7^(n) xZx Xf, 7^(n) = [r{™°^(2n), r}"'''^^)] ; 

1=1 
d 

?j{n) = n{n) X Z X Xf, n{n) = \{ [r^i^^\n), rf'^^'^ln)] . 

1=1 

Our last condition relates the choice of the vector r^™'^^\n)^ n > 1 and the kernel K with the 
parameter t appearing in Assumption 11. Let us assume that for any n > 1 

sup sup \K{u/r\ < lli^lloon"^ (4.24) 

re7^(n) |u|^(0,t]<* 

Note that (4.24) holds if K is compactly supported on [— t, t]'^ and A™^^){n) G (0, t)'' for any n > 1. 
Lemma 4. Assumption 11 and (4-24) ifnply Assumption 8. 

The proof of lemma is given in Appendix. Set for any r G (0, l]'^ and f > 

Mq^^,{r) = {[I2d + 1087V]C^ + 2.5g + 2v + 1.5) In (2V;r^) + C, 

where we have put C = IINS'"^ jlogg (Iblloo ||-^||oo)| + n ,R,d+i,d- 

Let 3 < ni < n2 < 2ni be fixed. Set Fj^^ [r,x^'^^) = max [F^^ (r, x^*^)) ,n2~^] and define 



Ai 
+A2 



Fn^ (r, xW) (nK-)-i Mg,v{r) + 2{v + 1) In (r, xW) } 



+ z 



{nVry^ In^ (n)j \Mg,vir) + 2{v + 1) In {F^ 



} 



+ z 



Theorem 9 below is the direct consequence of Lemma 2, Lemma 4 and Corollary 4. Remind that 



Cr(n,x^'^^) := sup 



Cr,i,xw{^ ) and N = {ni,. . . ,n2}. 



Theorem 9. Let Assumption 9 be verified and suppose that Assumption 7 (\\) holds with k = 
d + l,m = d + 2 and (i3d+i, Qd+i) = {2^, ['']")■ Suppose also that Assumption 7 (i) is fulfilled with 
{^d+2, Qd+2) = (^i)P^''^) C'lT'd Hrf_j_2,i = Xi, i G I, satisfying Assumption 11. Assume that (4-24) 
holds as well and if ^ n2 let (-^i)"^, i > 1, be identically distributed. 
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Then for any given decreasing sequences < r;^™"'^(n) < rl"'''^''\n) < I, I = l,d, n > 1, any 
b>l, q>l,v>l and z > 1 



(max) , 



rf< sup sup 

[ nSN (^,x{<*))67^(n)xXf 

Ef< sup sup 



4838e-^ + 2ni2-^|; 



"2 



niK 



V 



In^ 



n2) 



+ 2" + ^^ ( K.(min 



'r(min)(ni) 



ni 



2--V 



Remind that F^^ = sup sup (r,x('^M and the expression for the constant c' can 

neN (r,x('*))e:^(n)xXf 

be found in Theorem 7. We also note that the first assertion of the theorem remains vahd if one 
replaces the quantity Mg,„(r) by the smaller quantity (^[36d + 54:N]6-'^ + 2v + Qj In {2V-^) + C/2. 

But the corresponding upper function will differ from hi^'"'^'i) only by numerical constant. 

We also remark that < 2'^foo||5||oo^2 nf=i 2^'^^'^ for 

any n2 > 3 under Assumption 10 in 
view of Lemma 3. Moreover, if V^{min)(„-) > for some p > then Mq^y{r) can be bound from 
above by {\J2d + lOSA^JtJ^^ + 2.bq + 2v + 1.5)pln {2n) which is independent on r. Hence, if both 
restrictions are fulfilled the upper function JJ^^'^''^^ in Theorem 9 takes rather simple form, namely 



'ln(n) + z A2(g)[ln'^+^(n) + 



nVr nVr 
where the constant Xi{q) and A2(g) can be easily computed. 

Law of logarithm In this paragraph we will additionally suppose that Assumption 10 holds. 
Then, we remark first that statements 1*^ — 3*^ of Remark 11 hold. Next, we note that Lci+i{z) = z 
and L(i+2{z) = z^ in view of Lemma 2 that implies L'^^^iz) = ln{z) for any z > 1. Hence, the 
condition (4.17) is fulfilled with a = 1. 

Thus, all assumptions of Theorem 6 are checked and, taking into account that in our case 



Vt,w{n) = \\Cr{n)\ 



sup Cr{n, x^'^'') , 



we come to the following statement. 



Theorem 10. Let assumptions of Theorem 9 he fulfilled and suppose additionally that that As- 
sumption 10 holds. Then there exists T for any a > 4 



pj V^IICrHlL ^ 4840n5 

Pf<^ sup sup °° > 1 ^ < 

[ n>j r: n-i(lnn)-<K<l (^"1) V In In (n) J 

The uniform version over the set of bounded densities, similar to (4.22), holds as well. 
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The immediate consequence of the latter theorem is so-called " uniform-in-bandwidth consis- 
tency" : 



limsup sup " < T Pf-a.s (4.25) 

n^oo r:n-^\nnY<Vr<l J\n{Vr^^) Vlnln(n) 

The assertion of Theorem 10 and its corollary (4.25) generalizes in several directions the ex- 
isting results Einmahl and Mason (2000), Gine and Guillou (2002), Einmahl and Mason (2005), 
Dony et al. (2006) (see, the discussion after Theorem 8). 

We would like to conclude this section with the following remark. If K is compactly supported 
and 5(3,-) = g{-) for any 3 G 2^, where ^ is a bounded function, then all results of this section 
remain true under Assumptions 7 (i), 9 (i), 11, (4.20) and (4.21). 



5. Proof of Propositions 1-3 

We start this section with establishing an auxiliary result. Let £ be a set, di and d2 be semi-metrics 
on £ and let £ be a totally bounded subset of £ with respect to di and d2 simultaneously. Let 
N\{5)^ > 0, denote the minimal number of balls of the radius 6 in the metric dj, i = 1, 2 needed 
to cover the set C. 

Lemma 5. Let I G N* and dij,52j > 0, j = 1,1 be an arbitrary numbers. One can construct the 
finite subset L (^[61 J, 62,j],j =Tj^ := {h, ■ ■ ■ Jn} C C to^/j iV < 0^=1 ^1 ('^i,i/2) iV2 (<J2,i /2) and 
such that 

yieC 3ieL: di{i,i) < mm5ij, d2{i,£) < min52j. 



Proof of Lemma 5 Set Nij = N;[5ij/2), i = 1,2, j = 1,1. Since C is totally bounded 
di,i = 1,2 there exist L^^'-') = j^f'^^ . . • ,^5^/]} C C such that 

V£ G £ 31 e L^''^^ : di{£, i) < 2~^Sij, i = 1, 2, j = ~l. 

For any k = i = 1,2, put C^^f = |^ G £ : di^^,^''^'^) < ^^^^ij} and let C 

(1 7) (2 j) 

n First we note that for any j = 1,1 



m 



ki,k2 



^ = U U (5-1) 

ki=l k2=l 

Moreover, the construction of £[.^'"'^ and c!"^^^^ implies that 

di(/i,/2) < <5ij, i = 1,2, V/1,/2 G C^lk2- (5-2) 



Put M = 



{1, . . . Ni^j] X {1, . . . iV2j} j and define for any (k^^), . . . , k^')) G M 



'^k(i),...,k{o - n -^^^^ 



I r 
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The choice of an arbitrary point in each >C]^(i)^,,,_k(i) leads to the construction of L^[5i\6. 
1,1^ in view of (5.1) and (5.2). 

It remains to note that the cardinahty of J\f is equal to 0^=1 -^i (^ij/^j -^2 (<^2j/2 



5. 1 . Proof of Proposition 1 

I. Probability bound Fix s G 5a,b and put si ^ = si (2^1^^ and ■52, fc = S2 (2*^), /c > 0. For 

any A; > put 5i{k) = (24)-ixi2-'=/2si,fe, 62{k) = (24)-ix22-'=S2,fc and note that 5i{k),52{k) 
0, /c — )• 00 since si, S2 S §. 

Let Zk = L(^5i{k),52{k)\^ , /c > 0, be the set constructed in Lemma 5 with di = a, d2 = b, 

C = Q and / = 1. By N]^, k >0, we denote the cardinality of Zf^. 

Fix e > and put e = e/(l + e), ko = |_21n2 (l/e)J + 1. Let 6m, m = 1,. . . , A^'^q, be the elements 
of Zfcp . For any m = 1, . . . , N^^ define 

qM = e e . a(0,0^) < 5i{ko), hi9,em) < S2{ko)], 

and remark that the definition of the sets implies that = |Jm=i Q^"^^ ■ 
In view of the last remark we get 

p|sup^(xe) > f/l'^(?/,x,e)| < sup M'(xe) >C/f (2/,x,e)|. (5.3) 

Use J m=i Ueef™) J 

For any € let Zk{9) be an arbitrary element of Z^ satisfying 

4e,zk{e)) <6i{k), h{e,zk{9)) <62{k). (5.4) 

Fix m = 1, . . . , N)^^. The continuity of the mapping 9 ^ Xe guarantees that P-a.s. 

oo 

Xe = X6^+Y.[Xz,(e)-Xz,_,ie)], V0 G G^™), (5.5) 

A;=l 

where Zk^iO) = 9m, V0 G Q^"^\ Note also that independently of 9 for all /c > /cg + 1 

B{zk{9),ZkM9)) <5i{k) + 5i{k-l)=:6iik), (5.6) 
b(zfc(0), Zk-i{9)) < 52{k) + 52{k- 1) =: ^2{k). (5.7) 

This is the simplest consequence of triangle inequality and (5.4). Introduce the sequence c^, k > 1,: 

Cfc = 4"^ max {si^fc, si^k-i, S2,k, •S2,fc-i} 

and remark that ^f^yi c^. < 1 that follows from the assumption si,S2 G S- 
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We get from sub-additivity of ^, (5.5), (5.6) and (5.7) for any 6 G 9^"^) 

^ (Xe) < ^ (X9„J + sup sup cl^^iXu-Xv), (5.8) 

a(M,v)<5i(fc), h{u,v)<'52(,k) 

To simplify the notations we will write U instead of Ug^\y, x, 0) and £ instead of e^(x, 0). 
We obtain from (5.8) 



p| sup ^{xe)>u\ <F\^{xeJ>U{l+er'} 
[9ee("i) J 

oo 

+ J2 Yl P{'^{Xu-Xv)>eUck}=:Ii+l2. 

a,{u,v)<Si(k), h{u,v)<52{k) 

We have in view of Assumption 1 (i) 



(5.9) 



P{^{xeJ>U{l + £)~^} < cexp 



{A{9„,)y + {l + eyWB (6^) J 

< cexp < cexp {-(1 + er'{y + 2e-'£)} 

< cexp{-(l + e)-2y-e-2f}. (5.10) 

In order to get (5.10) we have first used that 5<i > sup^^g ^(^)) 5^2 > sup^^g B{6). Next, we have 
used that U > v, where v is the maximal root of the equation 

~2 , ~ =y + 2e-H. (5.11) 

We also have used that (l + e)~^ > 1/2. 

In view of Assumptions 1 (5), 2, (5.6) and (5.7) for any u,v G Zk x Zk~i satisfying a{u,v) < 
Si{k), h{u,v) < 62{k) we have 



P (x</.[n] - X^h) > ef^Cfc} < cexp' 



< c exp < 



{eUck}'' 



{a(n,t>)}^ + {eUck) h{u,v) 



{eUf 



(5.12) 



Here we have used that e < 1. Let us remark that 

5i{k)c;^ < 4(24)-ixi (2-^V2^i,fe + 2-('=-i)/2s^^^_^^ min {s^j, s"]. J < xi2-'=/2-i. 
S2{k)c^^ < 4(24)-ix2 (2-Ks2,k + 2-'=+is2,fe-i) min {s"],, s"],. J < X22-'=-i. 
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Thus, continuing (5.12) we obtain 

P ix^lu] - Xm) > eUck} < cexp | < cexp {-2^+ie^(j/ + 2e'^g) }(.5.13) 

Here we have used (5.11). Noting that the right hand side of (5.13) does not depend on u,v we get 



h < c J2 iVfciVfe-iexp{-2^^+ie2(2/ + 2e-2f) 

k=ko+l 

oo 

< cexp(-y) 7Vfc7Vfc„iexp{-2'=+2£:- 2^-^=0 }. 

k=ko+l 

Here we have used the definition of and that y > 1. 

Let us make several remarks. First we note that in view of Lemma 5 

ln{NkNk-i) < 2ln{Nk) < 2[(B^^ ^(^{2A)-^3<i2-^-''/hi,k) + ^^^y,[{24r'^22'''~^ 
Taking into account that si^k = si(2'^/^) and denoting Si = 2^^/^ we obtain from (1.9) 

(2:g^^(Si2-i-'=/2s, ,) =C;g^^(^3^,(485i)-isi(5i)) < dU'^^ {xi,e) = 2''eif {SfuQ) . 
Taking into account that S2,fc = •5i(2'^) and denoting 62 = 2'^ we obtain from (1.9) 



(5.14) 



(Sg ^^(x22-i-'=S2,fc) = eg b(x2(48(^2)"'52(52)) < S2ef^{Jc2,Q) = 2'=e(^^)(x2,e). 
Thus, we have for any k > 1 



2^^^8 



hi (iVfciVfc_i) < 2^+^ [eif (6) + eW(G) 
and, therefore, Mk > k^ + 1 

In {NkNk-i^ 
It yields together with (5.14) 

I2 < cexp |— y — 2e~^<S}. 
We get from (5.9), (5.10) and (5.16) for any m = 1, . . . , NkQ 



Vl sup ^ {xe)>u\ <2cey.Y>{-y{l + e)-'^ - e-'^e]. 

The last bound is independent of m and we have from (5.3) and (5.17) 

P i sup ^' ixe) >Us\< 2cNk, exp {-y{l + e)-^ - e-^£]. 

It remains to note that (1 — 3e)^ = (1 + e)~^ and that, similarly to (5.15), 

In (Affc,) < 2^^^ 8 < eH, 

and we come to the first assertion of the proposition. 
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(5.15) 



(5.16) 



(5.17) 



II. Moment's bound We get for any y > 1 



E:= sup1'(x^[e]) -U 
Keee 



) =q I x^^^P Jsup^'(x0) > C/ + xldx 
/+ -^0 [e^e J 

= q[U]'^ [ t-'-ip J sup^(x0) > (1 +w)C/ > d^;. (5.18) 
Jo [eee J 

Note that {l+v)U > ui^^ (^{l+v)y, 3i, 0^ . Therefore, applying the first statement of the proposition, 
where y is replaced by vy we obtain from (5.18) 

E < 2cT{q + 1) [(1 + efy~^U] " exp {-y/(l + ef}. 



5.2. Proof of Proposition 2 

We start with establishing some technical results used in the sequel. 

Preliminaries 1*^. First we formulate the simple consequence of Proposition 1. 

Let 6i, 02 be given subsets of G. For any s = (si, S2) G 5a,b and any x = (xi, X2) G \ {0} 
introduce the following quantity 

e^x, Gi, 02) = eif (xi, 0i) + e(^) (x2, 02) • 

Put any e > and any y > 

C/f (y, X, 01, 02) = xi^2[l + e-i]'e,-(x,0i,02) +y + X2 (2 [l + e"!] ©i, 02) + y) . 

Lemma 6. Let Assumptions 1-3 hold and let 0i,02 be given subsets ofQ. Let x be chosen such 
that xi > supgg0^ ^(^) c-n-d >^2 > supggQ^ B{6). Then Vs G 5a,b) Ve G (O, V2 — l] and Vy > 1, 

p| sup 'I'(xe)>C/f (y,x,ei,02)| <2cexp{-y/(l + e)2}. 
l6ieeine2 J 

Moreover, for any q> 1, putting Ce,q = 2cT{q + 1)(1 + e)^'^, one has 

e| sup ^{xe)-u^"\y,>^,ei,e2)y <Ce,, 

l6»eeine2 J + 

To prove the lemma it suffices to note the following simple facts. In view of the assumptions 
imposed on x and obvious inclusions 0in02 ^ 0i, 0in02 ^ ©2 we have 

xi > sup A{9), X2 > sup B{6), e^(x, ©1, ©2) > e^(x, ©1 n ©2) • 
6ieeine2 6'eeine2 

It yields Ug^^ (y, x, ©1, ©2) > Ug^^ (y, x, ©1 n ©2) and to get the assertion of the lemma we apply 
Proposition 1 with © = ©1 n ©2- 

2*^. Note that ^'(x») : x © — )■ M4. is obviously P-a.s. continuous in a V b as a composition 
of two continuous mappings between metric spaces. Hence Corollary 1 is applicable with T = ©, 
= a V b, (n,33,P) = {n,^,P) C(t,-) = '^(xe(-)) and ^(t) is either Yi'''\6) or \]'-g''''^\e), t = e. 
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-if/f (y,x,©i,©2 



exp 



(l + e) 



Proof of the proposition Put Si = {1 + eY, I > 0, and introduce the following sets 

= {eee: A61-1 < A{e) < asi} , = {eee: b6i-i < B{e) < b5i} , / g n*. 

The idea is to apply Lemma 6 with Qi = and G2 = for any given j, A; > 1. To do that we 
will need to bound from below vl^'^''(0) and ul^'^''^^(0) on G^^ H ■ We will consider only j^k 



such that 0^ n 0^ 7^ and supremum over empty set is assumed to be zero. Also we will accept 
the following agreement: if = 0, 6 = then A; = and 0^^ = 0. 

Probability bound Let 9 £ Q^j^ D 0^^ be fixed and put u = AeiO), v = Be{6). Note that 

efUAu,QA{Au)) > eif(Au,eA{A6,)) > e^f (^u,0?) > eif 0?) . (5.19) 



To get the first two inequalities in (5.19) we have used that 0^^ C ©^[Adj^ C ©^{Auj in view 

of 6j < u since 9 € 0^^- To get the last inequality in (5.19) we have used that the entropy is 

(7) 

decreasing function of its argument and that Jj+i > u since G Qa - 
By the same reasons 

efHBv,QB[Bv))>e(^UBv,QB[B5k))>efUBvM^^ (5.20) 



Taking into account that left hand sides in (5.19) and (5.20) are independent of 6, whenever 
6 G 05^'^ n we deduce from (2.3), (5.19) and (5.20) 



8(Ae{e),Be{e)) := et^(Au,QA{Au))+e^}'^i^Bv,eB{Bv)^ 

> eif fe+i, 0!^'^)) + ei^) (B5u+i,ef) = e, U Q^iK^f) , (5.21) 



for any 6 G qO''^), where we put x = [A5 j j^i, B_5k+ij . 

We obtain from (5.21), putting y = {l + ef z + e{5j) + £{6k) 



Yt'\e)>uf(y,xMi\Qf). V0G0?n0i^\ (5.22) 

Here we have also used that obviously {1 + £)'^A{9) > A5j+i =: >c\ and {X^ef'BiQ) > B_5k+i ='■ ^2 
for any G 0^^ n 0^\ Moreover, we have used that 2[l + e~'^]^ > Ae'"^ for any e G (O, \/2 - l] . 
Therefore, we obtain Vj, A; > 1 in view of (5.22) 



,(z) := sup \^ ixe) - V^;''\e)} < sup ^ {xe) - U^'^ (y, x, Q^, 0^^ 



Since xi := A6j+i > sup (j) A{9) and X2 := B_6k+i > sup (&) B{9), Lemma 6 is applicable with 

(i) (k) 

01 = 0^ ^-iid 02 = 0^ • Thus, applying it we obtain Vj, A; > 1 and \/z > 1 

P {"^Ikiz) > 0} < 2cexp{-z}w;j-Wfc, (5.23) 
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where we put Wr, 



l + mln(l + e)J 1 + ln{l + mln (1 + e)} 

oo 

^Wm<l + [ln{l + ln(l + e)} 



. Noting that 



771=1 

taking into account that the union of |0^^ H @^^\ j,k > 1, | covers G and summing up the right 
hand side in (5.23) over j, k, we arrive at the first assertion of the proposition. 

Moment's bound Using the same arguments having led to (5.22) and taking into account (5.21) 
we obtain for any 6 E O^'^^) and any q > 1 

where y = (1 + ef \z + T^es (x, og^) +(£ + <?) In {5j5k)\ , r, = 2(2 + e)e-\ 
Thus, applying the second assertion of Lemma 6, Vj, k >\ and Vz > 1 



sup ^{xe)-U'^sHy^^^^A^^f)] 



< 2cT{q + 1) 



{5j5k) ' ''exp{-z}(5.24) 



+ T,eg (x, e!^\ efj)) +ie + q) In {6^6^) 
Putting for brevity Cj ^ = eg ( x, ) and noting that for any j,k > 1 and z > 1 



U^J^(y,>i,Qf,ePj <2[xiVx2] [4e~^ej,k + y] < 2[A\/ B][5j+i V 6k+i] [2{1 + e~'fej,k + y], 
and that for any e G (O, -v/2 — l] 

2(1 + e'^ej^k + (1 + \z + r^Cj, ^ + (e + g) In {6j5k) 



z + TeCj^k + {£ + q) In {5j V 5k) 
we get finally from (5.24) for j,k > 1 

Ej^k < 2cT{q + l)[e-Hl + efY [A V BY{6,5k)~" e^p{-z}. 
Here we have used that {6j V 6k)/{6j6k) < 1 since 5j,6k > 1- Taking into account that 



(5.25) 



E(sup{^&(xe)-U(^ 



1 oo oo 

+ i=i fc=i 



and that X^^i Z^fcLi ('^j'^fc) ^ = [(1 + £)^ ~ l] ^, we come to the second assertion of the proposition 
in view of (5.25), where one can replace (1 + e) par 

25/2 

since e G (0, 1/2 — l] . We have also used 

that [(1 + ef - 1] < 2e-^ since e G {0,V2-l]. 
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5. 3. Proof of Proposition 3 

First we discuss the measurability issue. Note that ^'(x») : x G — )■ M+ is obviously P-a.s. 
continuous in a V b as a composition of two continuous mappings between metric spaces. Hence 
Lemma 1 is apphcable with T = G, c) = a V b, (fi, *B, P) = {n,^,F) C(t,-) = t = ^, 

3 = 2t, Tc = Qa and 5(3) = l](''''^\a), 3 = a. 

The proof of the proposition is similar to whose of Proposition 2. Let e € (0, 1) be fixed and put 
61 = {I + e)~', ^ > 0, 61 = 1, I < 0. Introduce the following sets: 

^? = {a E 21 : TiSi+i < n{a) < T161} , 21^'^ = {a E 2t : T261+1 < T2{a) < T25i} , Z E N. 

Fix j,k>0, a£ 2l[^^ n 21^^^'^ and put n = (1 + e)Ti(a), v = {I + e)T2(a). Note that 

> ^L.)(^rV(ri<5,-i),Q;(ri5,)). (5.26) 



'si(«,-) 

To get the first inequality in (5.26) we have used that @[{Ti6j^ C Q'^[u) in view of Ti6j < u since 
a E 21^"'^ To get the second inequality in (5.26) we have used that fl'A(^i'^j-i) ^ dAiu), since qa 
is increasing and Tidj^i > u for a E Moreover we have used that the entropy is decreasing 
function of its argument. 



Remembering the definition of ^9A{Ti^j-i) see (1.9), we have 



^%,.){K'9A{riS,^i),e'i{riSj)) = sup r^^^, ^^^^^^ jAr^5A(ri5,-i) (485)"^si(n, 5)) 
= supr^e . ^ (\Y'gA{ri6,^i){m''si {r^6„d) 

S>o ©il^i<5jj^a \^ lsi{Tidj,d)\J 



To get (5.27) we have used that 1 < u/Ti6j < 1 + e < V^, the definition of Ai and, as previously, 
that the entropy is decreasing function of its argument. We obtain from (5.26) and (5.27) 

^k-)(^^'^^^(^)'®i(")) ^ "i?(^i5,,) {9A{riS,-i),e[{rid,)). (5.28) 
By the same reasons we have 

^%,-){^2'9B{v),e'2{v)) > e^^|_^^^ ^ (5B(T24-i),e'2(r24)), (5.29) 

and, we get from (5.28) and (5.29) for any a E 21$^^ n 21^''^ 
£^'\a) := £'{u,v) > e^^|_^^ ^(5A(n'^i-i),e;(Ti<5,)) +e^^|_^^^ ^(^^ 

= eJx,e[{T,6j),e'2{T25k)), (5.30) 



where we have put X = ygA{TiSj^i) , gB{T26k-i)j and s = (^si (nJj, •) , S2 (r2(5fc, • 
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(?) (k) 

Note also that for any a G H Slg in view of monotonicity of functions and gs 

([l + e]Vi(a)) >gA{riSj.i) =: xi, ([1 + e]^T2(a)) >gA(r2<5fc-i) =: (5.31) 
Moreover, the definition of sets ©'i(-) and G2(-) impHes that for any a G 21^"'^ H Stj^^ 

e„ce;(ri(^,)ne'2(r24) 



and, therefore, for any a G 2tj^^ n Slg 



sup ^' (xe) < sup ^' (xe) . 

eee„ eee'i(ri5,)ne'2(T2<5fc) 



(5.32) 



Probability bound We get from (5.30) and (5.31) for any a G 2l[^^ n 21^''^ 

U(^'^'0)(a) > [/f (y,x,e;(ri<5,),e'2(r2<5fc)), y = {I + ef\z + Ro{Ti6„T26k 



where we have used that Rq is increasing, in both arguments, function. 
It yields together with (5.32) Vj, A; > 



sup 



sup <|*(xe)-U(^'^'°^(a)} 

* (Xe) - U^/^ (y, X, e; {ti6j) , e'2 (rs^^fc 



< sup 

6»ee;(ri<53)n0^(-2<5, 

Let us remark that the definition of the sets 0'^(-)) 0b(") the functions gA and as well as 
their monotonicity imply that 

XI := gA{Ti6j-i) > gAijidj) >g*A{ridj) =: sup A{e); (5.33) 

0ee;(-i5,) 

X2 := 5'B(T"2'5fc-i) > 5B(T25fc) > 5'B(T"2'^fe) =: sup B{9), (5.34) 

eeei,(-25fe) 

and, therefore. Lemma 6 is applicable with @i = Q'^ijidj) and ©2 = ©2(T25fc)- 
Thus, applying the first assertion of Lemma 6, we obtain Vj, k > and Vz > 1 

P > 0} < 2cexp {-z - Ro{Ti6j,T2Sk)}. 

Noting that the union of |2t]^^^ nSlg^^ j = 0, J,k = 0, K, | covers 21, summing up the right hand 
side in the last inequality over j, k we come to the first statement of the proposition. 

Moment's bound We get from (5.30) and (5.31) for any a G 21^"'^ n 212*^^ 

U(^'^^i){a) > C/(^)(y,x,e;(ri<5,),e'2(r2<5fc)), 

where y = (1 + ef + T,e,-(x, e;(ri5^), e'2(r24)) + Rg{TiSj,T26k)] , r, = 2(2 + e)e-K 
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It yields together with (5.32) Vj, A; > 



sup ( sup \^{xe)-U^''''''Ha)} 



< 



sup 



eee;(Ti5,)n0^T25fe)) 



Taking into account (5.33), (5.34) and applying the second assertion of Lemma 6, we have, analo- 
gously to (5.24), Vj, A; > and Vz > 1 



E(*f.'(-) 

<2cr(g+l) 



[/f (y,x,e'i(ri5,),e'2(r2<5fc) 



exp {-z- Rg{Ti5j,T26k)}. 



Z + TeCgi^X, @[ {ri5j) , Q'2 {T25k)j + Rq {riSj , r2(5fc) 

Putting for brevity ^ = (x, 0'^(ri(5j), 02(T2(5fc)) we note that for any j,k > and z > 1 

up (y, X, e; (ri5,) , G'2 {T26k)) < 2 [^a (n^,) V gB (r2<5fc)] [4e-2e,- ^ + y] 
Repeating the computation done after (5.24) we come to the second assertion of the proposition. 



6. Proof of Theorem 1 

Below ci,C2 . . . , denote the constants completely determined by d,p,fi and 7. We break the proof 
on several steps. 

1". Let Bg, 1 < q < 00, denote the set of functions vanishing outside and whose L^-norm is 
less or equal to 1. Later on the integration is always understood as the integration over W^. 
Consider the set of functions 

9/, = |0:M'^^M: e{x) = h''^ j Kt {t - x) i{t)dt, ^GB_p_, h £ Ti^. 
Put also 9 = Uhen^h and for any 9 £ Q introduce the gaussian process 

Xe= [ 9{x)b{dx). (6.1) 

In view of Young inequality Folland (1999), Theorems 6.18 and 6.36, for any 6 £ @ 

1 

/ f , ,0 \2 d{2-p) d{2-p) . d(2-p) 

110112:= / \e{x)\dx] <h^i^\\K\\j^<h^i^<(h'^'^"'^) 2P <oo, (6.2) 

since 2 < p < 00. Here we have also used that \\K\\q < ||i^||oo < 1 in view of assumption imposed 
on the function K. 
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Thus, the stochastic integral in (6.1) is well-defined and xe is zero- mean gaussian random function 
on such that 

v{e) = \\e\\2, ^(^1,^2) = 11^1-^2112. (6.3) 

We equip with the semi-metric p and in the next paragraph we compute the entropy of several 
subsets of 0. This computation allows us, in particular, to assert that Dudley integral is finite on 
0. It yields Lifshits (1995) that x» is P-a.s uniformly continuous on (0,/?), therefore Assumption 
2 holds. Moreover, we show that Assumption 3 is fulfilled. 

We conclude that Proposition 3 are applicable to xe^ on 0, with 0q, = 0/^, 21 = "H, a = \/2/3, 
A = y/2V. It remains to note that in view of duality arguments for any h ^Ti 

11^11 = sup f l{t)ih{t)dt, 

and, therefore, for any h gT-L 

Uhh = sup Xe- (6.4) 

2". In order to apply Proposition 3 we need to compute several quantities. First, we have to 
choose the function ti (since B,h = 0, hence T2 = 0). Set ti(/i) = and note that for any u > 

e[iu):= U 0^= U 

h: Ti{h)<u h>u~^ 

We note that computations given in (6.2) yield 

g\{u) := sup A{e) < V2u^^ =: gA{u), u > f/i^"^'^'^)) . 

3°. Let C;(")(5), 5 > 0, be the entropy of Q'j^{u) computed with respect to semi-metric a = \/2yO, 
where, remind p{-) = \\ ■ \\2- The following assertion is true: there exist cq completely determined 
by 7, d, p and p such that for any V7 E {d/2 — d/p, 7] and for any u > (/i'-™'^^)) 

g(")(5) < ci'u'^r'^/^, V5>0. (6.5) 

where ci = coS'^'^d'^. 

The proof of (6.5) is obtained by routine computations and it is postponed to the step 8*^. 
4". Choosing u = (/i^™^"^)) ^ (that yields Q'i{u) = 0) and 7 = 7 we get for any 5 > 



In view the condition 7 > d/2, Dudley integral is finite on and, therefore, x» is P-a.s uniformly 
continuous on (0,p). It complete the verification of Assumption 2. 

The last inequality shows also that there exist r > such that for any s E S, satisfying 
sup5>o 5^s~^('^) < 00, Assumption 3 is fulfilled. 
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5^. Let us now choose the function s. Set 6{u) = u'^/'^ "^/^ and for any n > 1 let m{u) S N be 
such that 2™(") < 5{u) < 2™(")+^ Define 



B{u,6) = {3/47r')(l+ log2(2-™(")j) 



We remark that 



-,{u, 2'=/2) = (3/47r2) [l + {{k/2) - m{u)) 



and, therefore, s(u, •) G S for any u > 1. Moreover, we note that \i p = 2 then s does not depend 
on u and it is given by 

5{5) = (3/4^2) (l+ [log2<5] 



Obviously the factor 3/47r^ can be replaced here by G/vr^. 

Now, let us compute the quantity Ai related to the function s. Remind that 



Ai := sup sup sup 



s(xt, 5) 



1 + log2 2—5 



where Zi = {h^™^^^) ^ > 1. It is evident that 

Ai = sup \ 1 \/ sup 

m>o I 5>o 1+ ^iog2 (2-"*-i,5 

21og2 (x)-l 



1 \l sup 



l + (log2 {X)f 
:>0 [l+(log2 {x/2))'' 



< 1 + sup 

x>0 



l+(log2 



1 + sup 



2y + l 



< 3. 



6°. Define 



d/2- d/{2p), 0<5<6{u); 
7, 6>6{u), 



and note that {d/2 - d/p) < d/i < j{6) < 7 for any S > 0. 
Putting C2 = Ci4(144)^ we get for any 5 > in view of (6.5) 



2d ( d(p-2) \ ( d 
C2UP V 2p J\jW 



5iu,6) 6'^/^^^^ =C2uT[5{u)f-Wl 5iu,6) 



(6.6) 



To get the last inequality we have taken into account that s(u, 6) < 1 for any u > 1, 6 > and that 
d/^{6) < 4. 
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We obtain from (6.6) for any 6 > 0, putting C3 = (47r^/3)^C2 
^-2gH(^A^i[5A(u)s(n,5)]/48(5) < C2U^ 

, 2/(p-l) 



/ § \rf/7(<5)-2 




U(n)J 


s(u, 5) 



2d 

< C2 U J' 



2d 

< C2 U P 



s(u, S) 



, , ^2/(p-l) / \'i-dh 



s(ti, (5) 



5c3 UP 2 



-a|log2 (2-'"(")5)| 



1 + 



log2 (2— 



where a = min {2/(p- 1),2 - d/7}. We obtain from (6.7) 



£'{u) := sup [r^e^") f [9a(^x)5(u, 5)] /485 
<5>o L ^ 



2d 

< C4 U P , 



(6.7) 



(6.8) 



where C4 = 5c3 sup 



x>0 



7^. Remind that ti(/i) = h ^ and, in particular, ti = [/i^™™^] Choosing e = \/2 — 1 we get 
from (6.8) putting C5 = 042°^/^ 

^(^-^)(/i) :=^:'f^ri(/i)) <C5h-f. 



Choose also Rr{t) = R{t) = t p (independent of r) that yields 

Ri^-^){h) := R(y2n{h)'^ = 2^/P/i-f . 
Choosing finally z = and putting U(/i) = \]^^'^'^'\h),e = \/2 — 1, z = 1, we obtain that 



r- / ..\/ -2d ,, _2d rf(2-p) d d 

U(/i) < V29A(2ri(/i)) V32c5/i p + 2'^/p/i p + 1 < ce/i p = cg/i-^. 



Let us compute now the quantities ■7^(^'°\ 7^(^•'?) defined in (2.10) with e = V2-l. 
Noting that = (/i("^'«))~^ and 2-^/2:,.^ < 2-i/2ti we get 



7^M) := ^exp|-(ri2-(-'/2)')"!>< 
j=o ^ 

:= X; [5A(ri2"^-/2) 
3=0 

qd{2-p) 

< eg (h^'^'^n exp 



C8 exp 



.2~3/2 / ^(max) 



-2d/p|_ 



exp <-[Ti2 



2d 

-072) A P 



2-3/2 / ^(max) 



-2d/p 



The assertions of the theorem follow now from Proposition 3. 
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'. It remains to prove (6.5). The proof is based on the fohowing inclusion: for any 7 G (0,7] 



e[{u) C B.%_ (7, 3'^Vdu') , Vn > 0, 



p-1 



(6.9) 



where ]H[*(-, •) C Mq{-,-) consists of functions vanishing outside of 

Let C;*(-) be the entropy of EI%_ (7, L) , L > 0, measured in ||-||2- It is well-known Edmunds and Triebel 

p-1 

(1996), that for any p > 1 there exist cq completely determined by 7, d, p and n such that for any 
{d/2 — d/p) < 7 < 7 and for any L > 

Since we consider only {d/2 — d/2p) < 7 < 7, (6.5) follows immediately from (6.9). Thus, we shall 
prove (6.9). 

Fix 9 € 0'i (u). By its definition there exists £ G B_p_ and h > such that 9 = * (■■, where 

p-1 

" * " stands convolution operator on W^. First, we note that all functions belonging to G vanish 
outside the cube ^2fi in view of assumption imposed on function K. 

Next, for any m = (mi, . . . , m^) G N"^ put \m\ = mi + • • • + m^, and set 7 = i + a and 7 = 1 + a, 
where /, / € N and < a, a < 1. 

Then, since K G ]HIoo(7, 1) we have for any m G N'^ such that |m| < / 



sup 



00. 



Above remarks allow us to assert that all partial derivatives 9^"^^ exist whenever |m| < / and they 
are given by 

0M(x) = [ {Khf'^Xt - x)l{t)dt, Vx G M'^, 



where for any function g the notation gi^™) or (if it is more convenient) (g)^"^^ is used for its partial 
derivative. 

We obtain in view of Young inequality for any A G M and any m G N*^ satisfying |m| = f 

(K/,)(™)(- + A)-(K,,)(™)(.) 



+ A) - 9^"^^ 



< 



p 
p-1 



< h- 



kM(. + [A//j])_i^M(. 



Here we have used that i G 



p-1 



We remark that if h < |A| then for any u £ either K'^'^Xu + [A/h]) = or K'-"^\u) = in 
view of the assumption imposed on the support of K. Thus, if /i < |A| 

|kM(. + [A//i]) - if('")(-)||^ < Wk''"'^^ < ||i^(™)||^(A//i)° < (A//i)", 
since h < |A| and lli^^"")]!! < H^^^^^^IL ^ ^ 

in view of assumption imposed on the function K. 
If /i > |A| then in view of the assumption imposed on the support of K we have 



kM(. + [A/h]) - K("*)(-) = [ ^ K^'^Xu + [A/h]) - K^'^^u) 



dn 



< 3° 



1=1 



a/2 



< 3'^Vd{\A\/hf. 
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Since h > u ^ we conclude finally that VA G 



(m)/ 



p 



(6.10) 



It means that 9 G (7, 3'^\/^u^ ) . As it was mentioned above all function belonging to vanish 



outside the cube 1^2^ that allows us to conclude that 6 G M* p [7, and, therefore, (6.9) is 

proved. 



p-i 



7. Proof of Theorems 3 6 
7.1. Proof of Theorem 3 
7.1.1. Preliminaries 



We start the proof with several technical results used in the sequel. Put for any i = l,n2, y G 
[ni/n2, 1] and a = b[ln (n2)] , 

Qiiy) = i(i/n.,i](y) + (n2y-^ + i)"iA,(y), Qiiy) = y-^Qi{y). 

Here we have denoted Aj = {{i — l)/n2, 1 ^ = 3, n2 and A2 = [l/n2, 2/n2] . 

For any o > 1 let \a\ be the smallest integer larger or equal to a. It implies, in particular, 
y G A|-n2j,] • First we note that for any y,y € [ni/n2, 1] and any i = 1, n2 

Q^{y)<l, Qiiy) = 0,yi> \n2y], \Qiiy) - Qi{y)\ < '^^ ^ IMv - vT ■ (7.1) 



The first and third inequalities imply for any i = 1, n2 and any y,y & [ni/n2, 1] 



my)-Qi{y)\ < {y/\yr' 

For any z,z' G M+ denote to[z,z'^ = ( 1 - 



|n2(?/ -y)r + ( 1 



1/2 



y ^y 
y^^y 



(7.2) 



zAz' 
zVz' 



, and remark that tt) is a metric on 



It follows from the relation \/^Xo(z, z') 



y I Kz) b(z') 



.1/2 



, where h is the standard Wiener 



process. Taking into account that > 1/2 and that Xo{y A y) < 1 we obtain from (7.2) 

my)-Q^{y)\<Se''[to{y,y)Y. (7.3) 
Here we have also used the definition of a. Taking into account that for any a < c 

sup p*^ (1- In (p))'= = 6'^-'= [c/a] ^ 

P6{0,1] 

we obtain from (7.3) for any 6 > 0, G [ni/n2, 1] and n2 > 3 

In (n2) 



sup \Qi{y) - Qi{y)\ < 8e 

i=l,n2 
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1 - In {to{y,y)) 



(7.4) 



Next, for any y,y £ [ni/n2, 1] 

my)-QM = 0, i^{rn2(yAy)l,...,rn2(yVy)l}. (7.5) 
We have for any y,y £ [ni/n2, l] in view of the first and third bounds in (7.1) and (7.5) 



f^my) - Q^iy\' < 



i=l 



2n2|y-y|, \ii2{y ^ y)^ - \My ^ y)^ > 
3 |n2(y - y)|'" , \T^2{y V y)l - \My A y)l < 2. 



To get the first inequahty we have also used that [n2(?/ V y)] — [n2(y Ay)] > 3 implies n2(y V y 
yAy) > 2 and, therefore, [n2(y Vy)] - [n2(y Ay)] + 1 < n2(y Vy - y Ay) + 2 < 2n2(y V y - y Ay) : 
2n2 |y — y| . Thus, we have for any y, y G [ni/n2, l] 



n-2 



. Y.\Qi{y) - Qi{y? < |y - y| + ^3 |n2(y -yT < 2^tt,{y, y) + 2^/3e'' [tt)(y, y)] °. 



Hence we get 



"2 



. 5ZlQ.(y)-Qi(y)l' < 8 VH^lD (y, y) +4^/3e^ [it, (y,y)]' 



ln(n2) 



1 - In (rD(y,y)) 



< 8VE^tt)(y,y) +4\/3e 
Taking into account that sup^>]^ z"^/^ [ln(2ez)]^ < (2b/e)^ we obtain 

tf (y, y) + \/374e(26/e)'' { 1 - In (tt,(y, y)) } 



"2 



\ i=i 



Finally we get for any y,y G [ni/n2, 1] and any b > 1 



"2 



. J] lft(?/) - < 8 
\ i=i 



2'' + 1 



{b)''^ 1 - In (tt) (y,y)) 



(7.6) 



y.i.S. Constants 



The fohowing constants appeared in the description of upper functions and inequahties found in 
Theorem 3. Let x = if ni = n2 and x = 1 if ni 7^ n2. 



CN,R,m,k = C)^^R „ fc + C)^^R „ + 2xa6, at = 2^' ln(2) + 2 sup (5^ A J)"! (965/^* (5)) ^ ; 



C 



NAm^k = sup 5 '{k 
<5><5. 



1 + hi 



/9216m52 



<^iv,ii,m,fc = sup 5 M A: 



, , /9216m(5\ 



+ 7V(m - k) 

+ 7V(m - k) 
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/4608mi?52 



log; 



2 ^ V 

/4608mi?5\ 



+ 1 



+ 1 



Put also Cd ■■= SUPj=o,fc+l,...,m SUp^g[o,i] D'j{z) 

Dj. Set at last, = Ay/2 [2^ + l] {b)'> and let 



V2, where Dj is the first derivative of the function 



Ai = 4V2^{^/C^V [xcb]) , As = (16/3) {Cd V 8e) , CD,b = ( V [xcb]) V [(2/3) [Cd V 8e)] , 

7.1.3. Proof of the theorem 
1°. Put for any i = l,n 

e{\),X,) =G{[),Xi) -EfG{t),X,), 
and define for any y £ (ni/n2, 1] and any i) £ Sj the random function 

n2 



^{y,i)) = n2-'Y.e{[),Xi)Q,{y). 



(7.7) 



i=l 



We remark that ^(,(p) = ^(p/n2,f)) for any p G N and any t) £ Sj. Thus, in order to get the 
assertions of the theorem it suffices to find upper functions for |^(-, •)! on [ni/n2, l] x io(n) in view 
of Assumption 5 and the definition of the number n. 

In view of Bernstein inequality Assumption 1 is fulfilled with ^ = h =: (y, {)) and (9 = h =: {y, f)) 



A\0) = AUh) ■.= 2n2-^Y.QKynfG\^,X,y, 

4 = 1 

r 

a'id, e) = a?(h, h) := 2n2^' Y.^f [ft(y)G(f), X,) - Qi{y)G{\),Xi 

i=l 

S(^) = Boo(h) = (4/3)n2"^[ su^ft(?/)l sup|G(f),x)|. 
b(0,0) =boo(h,h) := (2/3)n2-^ sup sup |e([), x)Qi(y) - e(^, x)Q,(y)| 



=1 n. ajeA" 



(7.8) 

(7.9) 

(7.10) 
(7.11) 



Note that af and boo are semi-metrics on [ni/n2,l] x Sj and ^(•,-) is obviously continuous on 
[ni/n2,l] X i3(n) in the topology generated by boo- Moreover, Af and i?oo are bounded and, 
therefore. Assumption 2 is fulfilled. 

Later on we will use the following notation: for any £2 : A" — )• M put ||n||oo = sup^.g^:^;' |£}(x)|. 
We obtain from (7.8)-(7.11) and (7.4) for any h, h G [ni/n2, 1] x i5(n) 



A}{\i) < 2(ni)-iF„,(fi)Goo(f)('=0, i?oo(h) < (4/3)(ni)-^Goo (fj^'^^) ; 
41n'^(n2)[„^,, , , 



(7.12) 



boo(h,h) < 



3ni 



|G(f), •) - G{i), OIL + 78eGoo 1 " 1^ v)) ' (7-13) 



where, remind, 7 = if ni = n2 and 7 = 1 if ni ^ n2. Here we have used that if ni = n2 the 
second term in the last inequality disappears. 
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We also get using (7.1) and (7.6) 
af(h,h) < V2n2'^\ 



Y,Q^iy)Ef[GH),X,)-G{f),X, 
\ i=i 



\ 



"2 



FnMGoo{¥''))Y,{Q^iy) - Q^iy)r \ 

i=l J 
< V2imry^y{F^,{l)) + F^,{fj))\\G{i),-)-G{hr)\\ 



+XCbJ2Fn, (^)Goo (^('=)) 1 - In (tt)(y, y)) 



(7.14) 



where we have put Cf, = 4^2 [2** + l] (b)^. Here we have used that if ni = 112 the second term in 
the last inequality disappears. 

For any r > put i^(n, r) = {f) G -^(n) : -Fn2(^) ^ ''"} • Onr first step consists in establishing 
an upper function for |^(-, •)! on H(r) := [ni/n2, l] xi^(n,r). As always the supremum over empty 
set is supposed to be zero. 

2°. Note that in view of (7.12) and (7.14) for any h,h G H(r) 



(7.15) 



af(h,h)<2V^(ni)-i/2<] J\\G{\:,,.) - G{l-)\\^ + xCbJG^im) 1 - In (tt,(y, y)) "(1.16) 



Moreover, in view of triangle inequality we obviously have for any h, h G H(r) 



af (h, h) < Mh) + Mh) < -\/8r(ni)-i Goo ( ll^'^)) V Goo (f)^'^ 



boo(h,h) < i?oo(li) + B^ih) < ^^""^ r 



3ni 



(7.17) 
(7.18) 



Set 



g(f,W,^('=))=Goo(^('=))vGoo(^^'-^^ 
We get for any f), f), satisfying g^'^^ ( t)^''^j V sup qj (t}j, t)0 < 1 in view of Assumption 4 (ii) 



j=k+l,m 

||G(f), •) - Gil OIL < Gd{g (f)^'^),^^'^) ( J^'^') + 



rn N 



j=k+i 



On the other hand, putting Lj{y) = Lj{y) V y, j = 0, /c + 1, . . . m, we have for any \], f), satisfying 



,(fc),'f,(fc) j^W) vsup 



j=k+l,m 



> 1 



|G(f),-) - G(f),-)||oo < ||G(f),-)||oo + \\G{^r)\L < 2a f)^'^ 



(fc) 



m N 

L,{g(f,('=),^(^))},,(f,,,^,)|- 

— 7^ I 1 
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i=fc+i 



Here we have also used that Cd > 2. Thus, finally we have for any f), fi 

^ m 



j=k+l 



The latter inequality together with (7.13) and (7.16) yields for any h, h G H(r) 



af(h,h) < a\(g{lj(''\i^^'^)g('\t,('\l^^'^)+ (O^^^^^'^) } fe,^ (7-19) 

^ ^ j=k+i ' 

+xy^7fJ(^V^[l - In (it)(y,y))] 

f m 

boo(h,h) < b]a(^('=\^^'-^)£.(^')(f)(^^^^'^)+ L,{g(f,W,^('^)^^ (7.20) 
+xa (f)^'^),^^'^) [l-ln(tt,(y,y))]"' 



where we have put a = 2^/r(ni) ^/^(a/C/j V [xcfe]), b = ^('^■p^^^J^^'^ {^2) ^ 

3°. We note that in view of (7.15) ) Assumption 1 (i) is verified on H(t) with 



A{e) = A (h) := a^Goo ((}('=)) , B{9) = B (h) := bG^ (fjC^)) , = f). 
The idea now is to apply Proposition 2 with = H(r). Put 

GM= inf Goo(f)(*^)), 



that yields -A = oy^G^ [r] and S = bGii[r]. Choose si = S2 = s* . To apply Proposition 2 one has 
to bound from above the function 



defined in (2.3). Here, in our ca-SG, a. — af , b — cind. 

Qa{Au) = {f) Gi3(n,r) : Goo{^^^^) < n'GM] x [ni/n2,l]; 
Qb{Bv) = {f) Gi3(n,r) : Goo(f)('=)) < vG^] x [ni/n2,l]. 

To compute the function £ let us make several remarks. 
S'^a. First remind that 



e[^'\Au,QA(Au) ) =sup(5-^e 



5>0 eA(A«),af 



(An(48(5)-is*((5)) ; 



,(boc) 



(^.,6^^.)) =suprie^^(^^,)_^^ (5.(485)-i.*(5)) . 
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We have in view of (7.17) and (7.18) that for any h, h G H(r) 

, boo(h,h) < b [Goo(f)(''0 VGoo(^^''^ 



af(h,h) < aJ Goo V Goo fl 



where we have also used again that Gz) > 2. Therefore, 



sup af(h, h) < Au, sup boo(h, h) <Bv. 
h.heeA (au) h,hee s [bv) 

It yields for any (5 < (^*, where remind 5* be the smallest solution of the equation (485)~^s*(5) = 1 







and, therefore 



e(?)(^n,e^(An)) = supr^^^^^^^^^^ {Au{mr^s*{5)) ; (7.21) 



'Qa [Au) 



65(^7;)) = snv5-^^^,.^^ (S^;(485)-ls*(5)) . (7.22) 



<5><5. 



3°b. For any t > 1 put S^\{t,rY) = {f)W ^ S)\{rY) : Goo({)(^)) < G^t} and note that the 
following obvious inclusions hold: 



QAiAu) C (n2G„[r]G„\ n) x io',!Vi x [m/na, l] ; 
&b{Bv) C {vG^[T]G-\n) X i^^+i x [m/na, l] . 



(7.23) 
(7.24) 



For any e > denote by '\R[^\e) the minimal number of ■'-balls of radius e needed to cover 
Sji(t, n) , ^j(e), j = k + l,m, the minimal number of ^j-balls of radius 6 needed to cover Sjj and 
let 91(e) be the minimal number of lt)-balls of radius e needed to cover [ni/n2, l] . 

Let H be an arbitrary subset of Sjf (t, n) x Sj"^^^ x [1/2,1]. It is evident that for any given 

> 0, ej > 0, j = k + l,m and e > one can construct a net {h(i), i= 1,1 [M]} C M such that 

Vh=(f),y)G]H 3iG {1,...,I[H]} 

< £,,(fj,,f5,(i)) < e,, j = k + l,7n, tt)(y,y(i)) < e; (7.25) 

m 

I[M] <ai(e/2)9lf) (e^'^Vs) JJ ^j(ei/2), VM C i^f (t, n) x i^^+i x [m/ns, l] .(7.26) 
Moreover we obtain from (7.19) and (7.20) for any u,v >l 

af(h,h) < a G„[r]n2£,(f)(f^('^),^('=))+ L,(G„[r]n2)^,,(f,,,y 



1/2 



j=fc+i 

~b 



+X^^VGj7][l-ln(tt)(y,y))J j, Vh,hGGA(^n); 
boo(h,h) < biG„[T]^;^,(f•^(f)W,^('^)+ ^ Z,(G„[r]^;)£>,(f),-,y 



, -6 

+xGn[T]t; ^l-ln(tt)(y,y)) 
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Vh,h G eA(Au). 



Thus, putting t = ti := u^G^[t]G^^ and choosing for any ^ > 



e = e 



we obtain in view of (7.23) and (7.25) with M = Qa{Au) 

WeQAiAu) 3iG eA(iin) } : af (h, h(i)) < c^. 

Putting t = t2 := vG_^[t]G^^ and choosing 

= I = i 



i/b 



(7.27) 



e = e 



we obtain in view of (7.23) and (7.25) with M = Qb{Bv) 

yh€QB{Bv) 3iG { [eB(^;;)]} : b^^ (h, h(i)) < 



(7.28) 



We get from (7.26), (7.27) and (7.28) for any <j > 



j = k + l 



4ma2Lj G„[r]'u2 



(7.29) 



+ <£[ni/n2,l],tt.(^2 ^expj - (2uay/Gn[T 
/ ^ \ m 

^ ^ ^ — J ^ j=A;+l 



~l"^[ni/n2,l],ra 

4°. We get in view of Assumption 6 

,2 



AmbLj[Gn[T]v 
2-^exp{-(2^bG„[r],-i)^/'}). 



(7.30) 



m 

i=fc+l 
m 

j=k+l 



4ma2L, GJr] 



< N 



Iog2 <^ 4a^mi2Lj (Gn[r]n- 



2\^-2 



AmbLj(G^[T]v 



^ [ 

=fc+i ^ 

<iV^ Mlog2{4bmi?L,(G„[r]i;)?-'} 



}]^ + x)-Pi) 



j=k+i 



+ 1 . (7.32) 



Taking into account that C;jni/n2,i],ro(") = 0) if i^i = ^^2) and ^[ni/n2,i],ro(£) ^ hi (2/^2) for any 
£ G (0, 1] and any n2 < 2ni, we have 



^[ni/n2,i],re 2 ^exp<^ - hua^y G^[t]<; ^ 



X ( 2 ln(2) + 2 ( 2^xa VGJ7]<^-M ^ (|';33) 



ni/n2,l] 



,„(2-i exp { - {2vbG^[r],-'Y^'' }) = x(21n(2) + 2 (2T;bG„[r],-i) ^ ) . 
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(7.34) 



Let us now bound from above ^^k,. (fe)- First we note that in view of Assumption 4 (i) 



'X:iJ(t,n),e;, 



Consider the hyper-rectangle Z{t) = [G^ „ 
the metrics 



X • • • X 

X • • • X 



{flfc G i^fc(n) : Gfc,n(f,fc) < . (7.35) 

[Gfc,ni *^A:,n] > ^ ^ li which we equip with 



m^'"' [z, z') = max |ln (zj) — In (z^') | , z,z' & ^{t), 

i=l,k 



where Zi,z^, i = l,k are the coordinates of z,z' respectively. It easily seen that for any <j > 



2{t),m('--) 



(?) < /c[lnlnt-lnln(l + ?)]^ < A;(ln(l + lnt) + [l+ln(l/? 



It yields together with (7.35) in view of obvious inequality (^f-t,. ^ (fe)(?) < ^z(t) mwi^/'^) 



We obtain from (7.36) 



(7.36) 



l-^^ < A:(ln(l + lnti) + [l + In {8ma^ G^[t]u\~^)] ; (7.37) 



< k (in (1 + In is) + [l + In (8mbG„[r]T;?^^)] 



(7.38) 



Putting Lj{z) = z-'^Lj{z) = max {z-^Lj(z), l}, we get from (7.21), (7.29), (7.31), (7.33) and (7.37) 

m 

ei?^ (^u, Qa{Au)^ < kS:^ In (l + In (tx^G^MG^^)) + N5:^ loga [ij (G^^n^) } 



+ sup 5 {k 

5>5, 



1 + ln 



/9216m(^2 



+X ( 2C^ ln(2) + 2 sup 5"^ (965/s*((^)) 



+ iV(m - A;) 



log2 



j=k+l 



+ 1 



i5>i5. 



In (1 + In {u^G^[T]a^^)) + iVC' ^ logs {^.- (G„[r]«2) } + C^J^^^^^, + 



j=k+i 



where, remind, a6 = 25^"^ ln(2) + 2 sup^^^^^ (5^ A 5)~^ [^Q5 / s* [5)) ^ . Note that ab < 00 since fa > 1. 
Repeating these computations we get from (7.22), (7.30), (7.32), (7.34) and (7.38) 



e^^°°) (pv, Qb {Bv)) < k5:' In (l + In (f G„[r]G„i)) 

m 

+N5:' log2 {Lj {G^[rW) } + Gi,')^^^^, + x^b, (7.40) 

j=k+i 

We deduce from (7.39) and (7.40) that £g, s = (s*,s*) is bounded from above by the function 
£{u,v) < M7^1n|(l + ln(n2G„[r]G-i))(l + ln(^;G„[r]G-i)) 

m r 



+N6:^ J2 log2 {Lj (G„[ 
j=k+i L 
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T U 



Li 



+ Gn, 



R,m,k • 



(7.41) 



Here we have used that 5* < 1. We note that (7.41) imphes in particular Assumption 3 and, 
therefore, Proposition 2 is apphcable with = H(r). 

5°. To apply Proposition 2 on = H(t) we choose e = -v/2 — 1 and bound from above the 
quantities 



P^_,(h) := 4[^-l]-'£:(^^2G„i[T]Goo(f)W),^/2G-i[r]Goo(f)(')) 
+2£(^2G„iMGoo(f)W)) +21 (V2G„iMGoo(f)('=))) ; 



+2(^/2 - 1 + g) In A/2Gn'MGoo(f)('=))^/2G„iMGoo(f)('=)) , 



where remind £(u) = In {1 + In (u)} + 2 In {1 + In {1 + hi (n)}}. 

Taking into account that l{u) < 31n{l +ln(u)},n > 1, [\/2 - l]~^ < 9 and that ^^[r] > G^ 
for any r, we obtain from (7.41) 

^v^-i(h) < [72M,^2^12]ln|l + ln(2Goo(f)('))G„i)| 

m 

+72iVC' E l°g2 {^i (2Goo(f)(')))} +36G;v,ii,m,A: =: 2P(f)('=)); 



jr=fc+l 



^^V2-i,.(h) < [144A;C' + 3(1 + g)] In (2G00 (fj^'^) G'^ 



+144Ar<572 ^ log^ |2 . (^2Go 

We remark that P et Mg are independent of r and y. 
Put for any 2: > and any h G H(r) 



+ 72G^,R,^,fc = 2Afg(fiW). 



V(^)(f)W) = 2^/2aJGoo(f)(^^))[^'(f)(^-))+^]+4bGo,(f,W) P(f)W) + 



U(^,.)((^(fc)) = 2y2a^Goo(f)W)[M,(f,W)+z]+4bGoo(f)('^))[Af,({)«) + 

where remind a = 2V^(ni)-V2 (^ v [xc^]) , b = '^''"^g'j^^"' 

We conclude that Proposition 2 is applicable with Vr^^ and Put for any n G {ni,ni + 

l,...,n2} 



o(n) = 2V2r(n 



, ^ , _ 8(Gd V8e)ln^(27i) 

0"'/'(yC^V[xc,]), b(n)= ^ ^ ^ 



and define 



V(^) (n, {)W) = 2V2a{n)JG^ (f)W) [P(f)W) + z] + 4b(n)Goo {i)^''^) P{i)^''^) + 



U^«)(n,f)W) = 2V2a(?i)^Goo(f)(^'))[M,(fiW)+z] + 4b(n)Goo(f)(')) [M,(f,('=)) + z 
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It is easily seen that a(n) > a, b(n) > b for any n G |ni, . . . ,n2} since n2 < 2ni. Therefore, 

It remains to remind that ^tj{n) = ^(?i/n2, f)) for any n G {ni, ni + 1, . . . , and any f) G All 
saying above allows us to assert that Proposition, 2 is applicable to |^(j(n)| on H(t) := {ni,ni + 

l,...,n2} xi3(n,T) for any r > with vi-^^(-,-) and ut^'^^(-,-)- 

Thus, putting h = (n, f)) we obtain for any r > 0, any z > 1 and any q > 1 



f S sup 
I heH(r) L 



|e^(n)|-V(^)(n,f,W) 



> ^ < 4 



1 + 



ln{l + 2"Mn2} 



exp{-z}; (7.42) 



Ef < sup 

I heH{r) 



|Cf,(n)| -U^«)(n,f,W)j I < 2(5«/2)+33g+4p^^ ^ [4 V " exp {-z}, (7.43) 



where, remind, A = a^y [t] and = bG^[T]. 

To get the statements of the theorem we will have to choose z. This, in its turn, will be done for 
Vt- and Ut- differently in dependence on the values of the parameter r. 

6°. Let r G N be fixed and for any r G N* put Tr = e^~^. For any r G N* denote io(r) = 
io(n,Tr) \Sj{n,Tr-i), ^(0) = io(n,ro) and let H(r) := {ni,ni + 1, . . . ,n2} x ^(r). 

Probability bound For any u > 1 put Zr{u) = ii + 21n(l + |r — r|) and remark that 



Zr{u) 



u + 21n(|ln(r,_i)|), r < r; 
n + 21n(l + |ln(Tr)|), r > r. 



We have for any r G N and any f} £ S){r) 

TO = i^n.,r(f}) zo(n) = ^x + 21n{l + |ln(i^„,r(f)))|}; 

Tr-l < Fn,(f)) =i^n„r(f)) z, (u) < + 2 hi | |ln (F„,r (f}) ) I } , 1 < r < r - 1; 

T. > Fn,(f)) =Fn„r(f}) z,(n) < + 2 In { 1 + |ln (Fn,,r(t))) I } , ^>r. 

Hence, we have for any r G N 

Zriu)<u + 2ln{l + \ln{Fn„rm\], Vf, G^(r), 
that yields for any r G N 

V(^.W)(n,[5W) < V^^(n,{)), V(n,f,)GH(r). 



(7.44) 



(7.45) 



Here we have also taken into account that < eFn2,r{^), Vf) G S){r) for any r G N. 

Thus, we get for any r G N and n > 0, taking into account (7.45), the inclusion H(r) C 11(7:^) 
and applying (7.42) with r = r^. 



sup 

.{n,f))eH(r) 



> ^ < 



1 + 



ln{l + 2-Mn2} 



exp {— n} 



[1 + |r - r[ 



(7.46) 
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Since obviously N x io(n) = U^QH(r), summing up the right hand side of (7.46) over r, we come 



to the first assertion of the theorem. Here we have also used that 16 



1 + 



ln{l + 2-Mn2} 



< 



2419 and the fact that S){n) C Sj{n) for any n G N in view of Assumption 5 and the definition of 
the number n. 

Moment's bound For any u > 1 put 

Zr{u)=u + 2ln{l + \r-r\)+qln (G„[r^]G-^). 
Similarly to (7.44) we have for any r G N and any f) £ S^{r) 

2,(n) < n + 21n{l + |ln(F„,,,(f)))| } + gin (G„[t,]G„1). 
Moreover, for any r G N by definition 

GM--= inf Goo((l('=)) 

and, therefore, for any f) £ S){r) 

Zr{u) < n + 21n {l + |ln (F„,,,(f)))| } + qln [g^o (i)^''^) On^} ■ 



Similarly to (7.45), it yields for any r G N 

^{zr(n),q) < i)), V(n, f,) G H(r). 

Note that for any r G N 



(7.47) 



AVB< 2CD,b 

where Cd,6 = (^/2C^V [7C5]) V [(2/3) (Cz? V8e)] . We get from (7.43) and (7.47), similarly to (7.46), 



V(ni)"iF„,G„ V ((ni)-i h/ {n^)G^] [GMQ.-' 



Ef < sup 

i(n,h)eH(r) 



|e(,(n)|-^Y^'')(n,fl)■ 



V(ni)"'^n.G„ V ((ni)-i In^ (n2)G„) 



[1 + |r - r[ 



where Kg = 2(7'?/2)+33g+4p^^ ^ l)(Ci),b)^. 

Summing up the right hand side of the last inequality over r we come to the second assertion of 
the theorem. 



7.2. Proof of Theorem 4 

For any / G N* set n; = j2', = {n/, + 1, . . . , and let 

Q = sup sup 



^Goo(fl('=))ln(l + ln(?i)), 
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We obviously have 



Pf{Cj >T} < J^Pf <^ sup sup 



1=1 



yfn r/[,(fc)(n) 



> T 



rf < sup sup 



Goo(f)W)ln(l + ln(n)) 
(n) - T^n-iGoo(tl('^))ln(l + ln(n)) 



>0>. 



Let / G N* be fixed and later on T,., r = 1, 2, 3 denote the constants independent on I and n. 
Note that in view of (4.11), (4.12) and (4.13) for any n e N; 



V, 



(21n(l+ln(n,0)) 




(n,f]) < AiW Fn-1 a 



P„ + 21n{l + |ln (F)|}+21n(l + ln (n)) 



+A2 [n-' In" (n) j Goo (fj^'O (^n + 2 In {1 + |ln (F)|} + 2 In (1 + In (n)) ) ; 

where we have put 

Pn = (36M-2 + 6) In (1 + b In (2n)) + SGA^J-^aln (^1 + In (^2^^ ) ) + ISC n ,R,ni,k{b) ■ 
Hence, for any n G N/ and any f) G i3(n) 



V, 



(21n(l+ln(n,)))^^^ < ^ / (f)^^)) In (l + In (n)) ^ 



n 



Goo(f)(''Oln^ (n)ln(l + ln (n)) 



n 



Since b > 1 can be arbitrary chosen and a > 2 let 1 < b < a/2. It yields for any n > 3 and any 
fjC') G ^^(n,a) 



Goo(()W)ln'' (n)ln(l+ln(n)) ^ / Gqo In (l + In (n)) 



n 



n 



and, therefore, putting T = Ti + T2T3 we get for any n G N/ 

,(21n(l+ln (n,))) 



^(n,f))<Tl 



Goo In (1+ In (n)) 



77, 



Noting that right hand side of the latter inequality is independent of and applying the first 
assertion of Theorem 3 with N = N;, r = and u = 2 In (1 + In (ti;)) we have 

Pf {Cj > T} < 2419 ^ (/ + In (j))-^ < 
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1.3. Proof of Theorem 5 

1^. We start the proof with estabhshing some simple facts used in the sequel. 

For any i G I let n(i) € N* and vrj(i) G I, j = 1, . . . ,n(i), be the pairwise disjoint collection which 
is determined by the condition: H^^i n H,m,k = 05 Vk ^ {7ri(i), . . . , vT"n(i)(i)} • First we have 

1 < n(i) < n, Vi G I, 

and we always put 7f„(i)(i) = i. It yields, in particular, that we can construct another collection of 
indices 7r(i) := {vrj(i) G I, j = l,n} given by 

^^^"l i, n(i) + l<j<n. 
Note also that for any 1 < j < n 

card(^ {i G I : 7rj(i) = p} ) < n, Vp G I. (7.48) 

Indeed, if card^ {i G I : 7rj(i) = p} ^ > n + 1 for some p G I, then 

card(^{iGl: H^,p n H„,i ^ 0} ) > n + 1, 
that contradicts to the definition of a n-totally bounded cover. For any i G I define 

H"^w= U U =U LJH^,., 

kel: H„,knH™,i^0 jel: H„jnH,„,k7^0 ^=1 i=i 

First we note that the definition of the set Sjm{') implies the following inclusion: for any i G I 

Sjm{i)m) ^ H„(i), Vf)„ G H^,i. (7.49) 

Next, taking into account that I]qgi lH™,q(f)m) < n for any f^.^ G i^m in view of the definition of a 
n-totally bounded cover, we obtain in view of (7.48) 



n n 



J]lH„(i)(f)m) < E^ZE^H ( ,a(^-) = EEE E 1h (p)(f)r 
iel iGli=li=l '"'-H-iWj i=l Z=l pel i:,r,(i)=p 



n n n n 



j = l 1=1 pel j = l 1=1 qel p:7rj(p)=q 



n n 



^ "'EEElH,n.Jf)m) <n^ Vf)™Gio„.. (7.50) 

i = l 1 = 1 qel 



Define finally for any i G I 

n2 



fi := ni~^ / /i,j(x)i/i(dx) 
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and let Ii = {i e I : fi > (ni)"^} and I2 = I \ Ii- 

2". Let us fix i G Ii and for any n > 1 define Hi(n) := i5i(n) x ^^^i x Hm,i, i G I- The idea is 
to apply Theorem 3 to {Hi(n), n > 1} that is possible in view of Assumptions 7 (i) and 5. To do 
it we first note that the definition of Ii together with (7.49) implies for any n G N 

(M >ln(l/fi), Vf)™GH^,i. 

It yields for any n G N and any f) G Hi(n) 

V^^^(n,W > V^")(n,f,), Z^i^'^'^^Kf,) >a^"'«)(n,[)), 

where u = In (l/f;) + z. We deduce from Theorem 3 for any i G Ii 



< sup sup 

IneN fieHi(n) 



Ef < sup sup 

I nSN f)eHi(n) 



|ei,(n)|-V^^^(n,[)) 



> ^ < 2419 fi e"^; 



< fi A„(ni,n2)e'^ 



where we have put Ag(ni, = Cq ^J {rixY^F-a-^Cl^ V ((ni) ^ In'^ (n2)Gn) 
We have in view of (7.50), taking into account that n2 < 2ni, 



n2 » r 

^fi = (ni)-i5^ / /i,(x) ElH.(i)(^) 
iei i=i iei 



VI (dx) < 2n^ 



Putting fp-\n) = Uigi Hi(n), n > 1, we obtain from (7.51), (7.52) and (7.53) 



:"f < sup sup 



Ef < sup sup 



|e.,(n)|-V^"'^)(n,f,)' 



%{n)\-U^''^'^''\n,\)) \ <2A,(ni,n2)n5e-^ 



> ^ < 4838 e'^^; 



(7.51) 
(7.52) 



(7.53) 



(7.54) 
(7.55) 



To get (7.55) we have used obvious equality: [sup^, Q(a)]^ = sup^, [Q(q;)]^. 

3". Fix i G I2 and note that in view of Assumption 8 for any n > 1, any f) G S){n) and i > 1 

<Goo(f)(^^) [Pf{^M <Goo{^^^^) [Pf{Xi,iGH^(i)}+n-i . 

The last inequality follows from (7.49). It yields for any n G N and any f) G S){n) 

n 

J^Ef |G(fi,X,)| < Goo(f)('=)) [fi + n^^] <2(ni)-iGoo(f)('=^), (7.57) 
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i=l 



since fi < (ni) for any i G I2 and v > 1. 
Introduce random events 



. i=l 



iei2 



Note that if the random event C holds (where, as usual, C is complementary to C) then for any 
n € N and any t) € Sj{n) in view of Assumption 8 and (7.49) 



n 



'i^|G(f,,X,)| < 2n-iGoo(f)(')) <2(ni)"^Goo(f)('0. 



(7.58) 



i=l 



Taking into account that bounds found in (7.57) and (7.58) are independent of i we get for any 
n G N and any f) E ?)^'^\n) := ?){n)\ £ ^^^^n) 

\Un)\-i-c < 4(ni)-^Goo 
Noting that for any I) £ Sj, z > 1 and n G N 

V^^'^)(n,f,) > 8n-iGoo(f)('))>4(ni)-iGoo(f)('=)), 
Z7('^'^''^)(n,{,) > 8n-iGoo(f)(''))>4(ni)-iGoo(f)('^)). 
and, therefore, if the random event C is realized we have 



sup sup 
It yields, first. 



k(,(n)|-V('^'^)(n,f)) 



Pf < sup sup 

[neNf)Gfl(2)(n) 



< 0, sup sup 

neN f)e5(2)(n) 



k(,(n)|-Z^^^''')(n,fi) 



< 0. 



l^hNl -Vr''(n,f))J >0^ <Pf{C} < ^Pf{Ci}. (7.59) 

J iei2 



Next, taking into account the trivial bound |'?f,(n)| < 2Gn for any nGN and any t) G ^{n), we get 



Ef < sup sup 

[n6N 1)6^(2) („) 



Ci,in)\-U^^'''''\n,i)) \ <(2G„)>f{C}<(2G„)^^Pf{CiX7.60) 



iei2 



For any i G I2 put pj^i = P/|Xi^j G Hm(i)|. Since Xi^i, i > 1, are independent random elements 
we have for any i G I2 and any A > in view of exponential Markov inequality 

Pf {Ci} < exp |-2A + (e^ - 1) £p*.i| = {"^^ + ^(e^ " 

Minimizing the right hand side in A we obtain for any i G I2 

Pf{Ci}<(e/2)2(nifi)2<2fini2-^ 
The last inequality follows from the definition of I2. We obtain finally in view of (7.53) 

Pf {Ci} < 2n5 nl2-^ (7.61) 



i6l2 



The assertions of the theorem follow now from (7.54), (7.55), (7.59), (7.60) and (7.61). 
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7.4- Proof of Corollary 4 



To prove the assertion of the corollary it suffices to bound from above the function £„„(•). Remind 
that we proved, see (7.56), for any n > 1, any G S){n) and i > 1 



It yields for any n G N and any I) G S)(n) 

n „ 

Fn^ (i)) < Goo (f)^*^^) [^n([)m) + n^'] , = / /l.iC^;)^^! (dx) . (7.62) 

j — l -'-f)m(flm) 

Indeed, if ni = n2 then n = n2 and (7.62) is obvious. If ni / n2 then Pf|Xi^j G -^■m(f)m)| 
is independent of i since we supposed that Xi^i, i > 1 are identically distributed. Hence, A„(-) is 
independent of n and (7.62) holds. Let n G N be fixed and let G Sj{n) be such that F^^ (f)) > 
If A„(f)m,) < n"-*^ we have Goo{i}^^^) > 2~^^/n and, therefore, 

2v\ In {2Goo (fl^'^)) } I > V ln(n) > i2n,.(f)^). 
If An{i}m) > we have F^^il)) = Fn^{l)) < 2Goo (f)^'')) A(f)m) and, therefore, 

£n,.(f)„^)<ln(A;^t)«^)) < In (2G^ Hj^"^) p-^Hl))) =|ln(2Goo(f}('=))Fn~'(f)) 

< ln(2Goo(fl('=)))| + |ln(Fn,([))^ 
Here we have also used that An{i}m) < 1- Thus, if F^^i^) > n~^/^ for any v >l 

£n,.(f)m) <27;|ln(2Goo(f)('=^))| + |ln(F„,(f)))|. (7.63) 
If now I) G ^(n) be such that F^^i^) < then obviously Fn^i^)) < n"^^'^ and, therefore. 



In Fn,(f)) > t;ln(n) > £„,„(f)^). 



2?; 



The latter inequality together with (7.63) yields for any n G N, any f) G ^{n) and u > 1 

£n,.(t)rn) < 2t; [ |ln (2G00 (fl^^^^)) | + |hi (f„, (f))) I ] . (7.64) 

Hence, choosing r = ln(n2) and replacing £«,!'(■) in the expressions of Vr'''^'*(-, •) and l/(r^'^''^\-, •) by 
its upper bound found in (7.64) we come to the assertion of the corollary. 
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7.5. Proof of Theorem 6 

For any / G N* set ni = j2', = {ni,ni + 1, . . . , n/4.1} and let 



Cj = sup sup 

"^j f)W65j(n,a) ^/Goo(f)('=)) In {Goo V In In (n) 

We obviously have for any y >0 
Pf{Cj>T} 



< ^IPfS sup sup 



1=1 



neN, 



r/^W (n) - rJn-^Goo In {Goo } V In In (n) 



f)(fe)6X:ii(n,a) 

Remind, that for any 3 < ni < n2 < 2ni and any n € N 



> 



V(^'^)(n,f)) = Aiy(F„,(f,)n-i)Go 



+A2f?i^Mn^ (n))G, 



P,(flW)+2(^; + l)|ln{Fn,([))}| + 
P,(f)('=))+2(t; + l)|ln{F„,(f))}| + 



2; ; 



Let Z G N* be fixed and choose v = 3 and z = 21n(l + ln(?i;)). Later on T,., r = 1,2,3,4 denote 
the constants independent on I and n. ^ 
We have in view of (4.11), (4.12) and (4.17) for any n E and [} G S^{n) 



p(3,21n(l+ln(n,)))(^^f^) < Ti^ 



Goo ({)W) [in {Goo (flW) } V Inln (7i) 



+ T2 



n 



Goo(f)('=))ln''+i (n) 



n 



To get the latter inequality we have used, first, that 

FnM\^n{FnAi))}\< sup x|ln(2;)| =:c(F) <oo, VF < 00. 

a;e(0,F] 

Next, to get the second term, we have used that for any n € and f) G Sj{n) 

Psih^'^) < T3ln(?i), |ln{Fn,(f))}| < max [| In {F} | , ln(n,+i)] < max [| In {F} | , ln(2n)] . 

Since 6 > 1 can be arbitrary chosen and a > 4 let 1 < b < a/2 — 1. It yields for any n > 3 and 
any f)^'^) G S^iin, a) 



Goo(f)«)ln''+i (n) 



n 



G„o(l)<'l) ln{G„o(l]W)} Vlnln(n) 



n 



and, therefore, putting T = Ti + T2T4 we get for any n G N; 



p(3, 2 In (1+1: 



Goo(f)(*^)) ln{Goo(f)W)} Vlnln(n) 



n 
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Noting that right hand side of the latter inequahty is independent of and applying the first 
assertion of Corollary 4 with N = and z = 2 In (1 + In (n^)) we obtain 



{OO CX) ^ ^ 

2419 5] + In + 2-' < 2n' 

1=1 1=1 J 



2419 . 4840n^ 



ln(j) J ln(j 



8. Appendix 

Proof of Lemma 1 l''. We start the proof with the following simple fact. Let T be an arbitrary 
subset of X. Then 

supC(t, •) ^ — measurable. (8.1) 

Indeed, since T is totally bounded T is totally bounded as well. Denote by T the union of 2~'-nets, 
/ > 0, in T. Let CIq = jo; G : C(")'^) is continuous} and let be the complementary to Qq. 
We have for any x G M 



uj € ft : sup C(t, uj) < X > n Clo = <uj € fl : sup C(t, w) < x ^ n fio G ^ 

since T is countable dense subset of T. It remain to note that |a; G fi : sup^^^ C{^^^) ^ x| fl fio £ 
*8 since P(rio) = and the considered probability space is complete. 

2°. Set 3(n, A;) = {3 G 3 : £^(3) G [k/n, (A; + l)/n]|, n G N*, A; G Z, and let K(n) C Z, n G N*, 
be defined from the relation if /c G K(n) ^{n, k) / 0. Put also T(A;, n) = U^£3(fc ^jT^ and define 

^k,n{i^)= sup C(t,w) - (fc + 1)/"-, Cn(w) = sup Cfc,n(w). 

%,n(^) = sup sup C(t, •) - 5(3)] , ri{uj) = sup sup C(t, uj) - 5(3)] . 



363{fc,n) 



tex, 



Some remarks are in order. First, the definition of 3(^, n) implies that for any k G K(n), n G N* 

6,n(-) <^m(-) <efc,n(-)+^"^- (8.2) 

Next, taking into account that 3 = UfcgK{n)3(^, ^t-) for any n G N* we have 

r?(-)= sup %,„(•), VnGN*. (8.3) 

A:6K{n) 

We obtain from (8.2) and (8.3) that for any n G N* 

< ??(•) - Cn(-) < sup [r?fe,n(-) - 6,n(-)] < 
feeK{n) 

and, therefore, r/(-) = limn-s.00 Cn(-)- It remains to note that ik,n{-) are *B-measurable for any 
k G K(n), n G N* in view of (8.1), that implies obviously that ^n{') is ^-measurable for any 
n G N*. Thus, r/(-) is *B-measurable as a pointwise limit of ^-measurable functions. 
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Proof of Lemma 2 l''. Remind that 

Goo(r) = V-'\\g\U\K\U G„ = i;,7i.)(„)||5lloo ||i^||oo, n > 1. 

Hence we have have for any I = l,d and any [)i := n G r|™™^(n), r^^™'*^''(n) 



(8.4) 



G'/,„(n) = ||5lloo||i^||c 



G,„=||ff||oo||i^||c 



,^(min) 

-1 r 



(min) / N ; 

rl '{n)/ri 
r^™"''' (n) I rf^'^^'^ (n) 



^(min) 



7i 



Thus, we get for any n > 1, r G iKn) and for any j = l,d 



n 

1=1 



(max) 

i 



> 



(max) 



Gj,n ('"j 



We conclude that Assumption 4 (i) is fulfihed. 
2". Remind that for any r, r' G TZ{n) 

QnHi^^i^l ■■= raaxmo[G^n{ri),Gj^n{r'i)) = maxji ln(n)-ln(r[) 



l=l,d ^ 



1=14 



(8.5) 



3°. Set \\Kr - KA 



sup^gjjd |i('r(-2) — Kri{z)\ and note that for any x G x X^+i and for 



anyf) = (r,3,2/W),f)'=(r',3',zW) 

\G{\),X) - G(f)', X) I < II^IU \\Kr - ifr'lL + ll^lloo % V K']"' 1^(3, x) - 5(3', x) 

+ ||5||ooV;7^ \k (/?(xW,yW)/r') - K (p>W,zW)/r') 



< 
+ 



/\1 a 



,1/7^ |a' (p(xW,y('^))/r') - A' {p[x^''\ z'^^^) / r') 



The get the last inequality we have used Assumption 9 (ii). Using Assumption 9 (i) we have 
k(p{x^'')J''))/A -k(p{x^''\z^''^)iA < L,ui^[{r[)-^\pi{xuyi) - pi{xuzi)\] 

^ ^ ^ ^ l=l,d 

< Lima^[{r'i)-^pi{yi,zi)] . 

l=l,d 

To get the last inequality we have taken into account that pi, I = l,d, are semi- metrics. Note also 
that {r'i)~^ < V^^ for any I = l,d, since r[ < 1 and we obtain 

A:(p(xW,y('^))/r')-A:(/?(xW,z('^))/r')| < L.V;;' p^''\y^^) , z^"'') , (8.6) 

where we have put p^'^^ = max^^j-^ pi . Obviously, 

lli^r- - < \\K\\^ \vA - v;;' \ + [Vry Kr' \\k{ ■ /r) - k{ • //) ||^ . (8.7) 
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We have in view of Assumption 9 (i) and (8.5) 



\K{- /r) — • /r') < Li sup max 



1=14 



Li 



exp < max ln(n) — ln(r;) [■ — 1 



\ui\\l/ri-l/r\\ 
l + \ui\ Al/r[) 



< Li max 



V 

r/ A r( 



exp<j7 (^r, r'^ I 



we we have put 7 = min[7i, . . . , 7^]. Moreover, we obviously have for any r, r' E (0, 1]' 



14 A ~ Vr 

Thus, we finally obtain from (8.7) 



= exp i V I In (n) - In (r[) I < exp 

VrAr' [ 1 J 



r, r 



Ili^lloo (exp{d^,W(r,r')}-l)+Li 



exp<j7 ^^'^'^^(^r, r'^l 



This yields together with (8.6) for any = (r, 3, y'-'^^) and f)' = (r', 3', z^'^^) 



sup |G(f),x) - G(f)',x)| 

x6XfxXd+i 



(8.8) 



< 



lli^lloo exp 



^ r') I 



1 +Li 



exp 



+LjK\\^ [Vr V Vrr' HhiT + ^ill^llooK^V'Hy^'^ ^^'^) 



< ||5||oo||i^||oo[K-AK.]-^ 



Do [q^''^) + D,+^{q,+^) + [VrAVrr'Dd+2{p^''\y^''\z^''^)) 



where we have put ^^+1 = [t)]°, Dd+i{z) = {La/[[g[[oo)z, Dd+2{z) = {Li/[[K[[oo)z and 
Do(^) = exp {dz} - 1 + (Li/||K||oo) ( exp {^-^z] - 1^ 



Putting L(i+i{z) = z and Lii+2{z) = z^ we obtain from (8.4) and (8.8) for any \) = (r,3,y^'^^) and 
f)'=(r',3',zW) 

sup |G(f),x)-G(f)',x)| <Goo(r) VGoo(r')I?o(^^''H^'^' 



z^ 



We conclude that Assumption 4 (ii) is fulfilled. It remains to note that if Xf consists of a single 
element then last summand in the right hand side of the latter inequality disappears that correspond 
formally to Ld-\-2 = 0. This completes the proof of the lemma. 
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Proof of Lemma 3 In view of (4.21) for any r G (0, 1]'^ 



< f, 



oo lli/lloo 



Kr /iW(dx('^)) =: fooll^llooX.. (8.9) 



Denote for any I = l,d 

Dii{ki,ri) = Mi(2'''+\,xi^ \Mi(2'''ri,xiy $Hz(0, n) = (n, x,) , kGN. 

and for any multi-index k = {ki, . . . , kd) € N*^ set 9^k,r = ni(A:i, ri) x • • • x d\d{kd-, r^).. We get in 
view of Assumption 10 that = IJkeN'* ^k,r for any r G (0, l]'^ and, therefore, 



We note that for any k G N'^ that for any x^"^) G 9^k.r 



Kr (p(xW,xW 



K 



id) ^{d)^^' 



<V-^ sup \K{u)\ =V-^K{t{k)). 
I«l^nt(k) 



where, we have put t(k) = (2^^!, . . . , 2'^'') and where, remind, Ilf = [0, ti] x • • • x [0, td], t G 
Thus, we obtain from (4.20) of Assumption 10 (remind that /x^*^^ is a product measure) 



ir < E ^(*(k))^w (nt(k)) < i^(t(k)) 



1=1 



< 



.1=1 



Jj2^'' 



i=i 



^.10) 



We get finally from (4.19) of Assumption 10 that for any r G (0, 1]'' 

d 



Ir < 2% Jj2^'L('). 



1=1 



The assertion of the lemma follows now from (8.9). 



Proof of Lemma 4 Remind, that for the considered problem 

For any x^"^^ G Xf and any r > denote B^td, (r, x^"')) = {x^"') G Xf : p^'^) (x^, x^'^)) < r} where, 
remind, p^'^^ = max[pi, . . . ,pd]- The following inclusion holds in view of Assumption 11 



„(.,(t,xW) CXf(x('^)), Vx^'^) GXf. 
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^.11) 



Indeed, suppose that 3?/('^) G B^(d) (t, x^'^'^) such that ^ Xf (x^'^)). Then, the definition of Xf (x^^)) 
implies that for any p, q G I such that x^^') G Xp, y^^^ G Xq necessarily 



XpHXq 



Hence, in view of Assumption 11, p^'^'> [y'^'^\x^'^^^ > t and, therefore, y^'^^ ^ ©^(d) (t, x^'^)) . The 
obtained contradiction proves (8.11). 

Note that in view of Assumption 9 (ii) for any x G Xf x X^+i and any \] = (r, 3, x^'^)) 



\G{\),x)\ < ii5iioo^r'|^( 



id) 



and, therefore, we get from (8.11) and (4.24) 

sup \G{^,x)\ < \\g\\ooV~'^ sup 

< 1151100 sup sup \K{u/r)\ < ||sr||oo||-?^||ooK~^"-~"^ 



K (p{x^'^\x^'^^)/r 



n 



-^Goo(r) = n-iGoo(f)(')). 



References 

Alexander, K. S. (1984). Probability inequalities for empirical processes and a law of the iterated 

logarithm. Ann. Probab. 12, 1041-1067. 
Barron, A., Birge, L. and Massart, P. (1999). Risk bounds for model selection via penalization. 

Probab. Theory Related Fields 113, 301-413. 
BOBKOV, S.G. (1988). Upper functions and oscillating Gaussian processes. J. Soviet Math. 43, 6, 

2745-2751. 

BOUSQUET, O. (2002). A Bennett concentration inequality and its application to suprema of em- 
pirical processes. C. R. Math. Acad. Sci. Paris 334, 495-500. 

Cavalier, L. and Golubev, Yu. (2006). Risk hull method and regularization by projections of 
ill-posed inverse problems. Ann. Statist. 34, 1653-1677. 

CoiFMAN, R.R. and Weiss, G. (1971). Analyse harmonique non-commutative sur certaines espaces 
homogenes. Lectures Notes in Math. 242, Springer- Verlag, Berlin. 

COULHON, T. , Kerkyacharian, G. and Petrushev, p. (2011). Heat kernel generated frames 
in the setting of Dirichlet spaces. Manuscript. 

DONY, J. , EiNMAHL, U. and Mason, D. (2006). Uniform in bandwidt consistency of local poli- 
nomial regression function estimators. Australian Journal if Stat. 35, 105-120. 

DONY, J. and Einmahl, U. (2009). Uniform in bandwidt consistency of kernel regression esti- 
mators at a fixed point. IMS Collection 5, (Highdimentional probability. The Lumini volume), 
308-325. 

Edmunds, D.E. and Triebel, H. (1996). Function Spaces, Entropy numbers and Differential 

Operators. Cambridge University Press 
Egishyants, S.A. and Ostrovskii, E. I. (1996). Local and global upper function for random 

fields. Theory Probab. App. 41, 4, 657-665. 

71 



EiNMAHL, U. and MASON, D. M. (2000). An empirical process approach to the uniform consistency 

of kernel-type function estimators. J. Theoret. Prohah. 13, 1-37. 
EiNMAHL, U. and Mason, D. M. (2005). Uniform in bandwidt consistency of kernel- type function 

estimators. Annals Stat. 33, 3, 1380-1403. 
FOLLAND, G. B. (1999). Real Analysis. Modern Techniques and Their Applications. Second edition. 

John Wiley &: Sons, Inc., New York. 
GiNE, E. and Koltchinskii, V. (2006). Concentration inequalities and asymptotic results for ratio 

type empirical processes. Ann. Prohah. 34, 1143-1216. 
GiNE, E. and ,Guillou A. (2002). Rate of strong uniform consistency for multivatiate kernel 

density estimaton. Ann. Inst. H.Poincare, Proha. Statist. 38, 907-921. 
GiNE, E. and Zinn, J. (1984). Some limit theorems for empirical processes. Ann. Prohah. 12, 

928-989. 

GOLDENSHLUGER, A. and Lepski, O. (2008). Universal pointwise selection rule in multivariate 

function estimation. Bernoulli 14, 1150-1190. 
GOLDENSHLUGER, A. and Lepski, O. (2009). Structural adaptation via Lp-norm oracle inequali- 
ties. Prohahility Theory and Related Fields 143, 41-71. 
GOLDENSHLUGER, A. and Lepski, O. (2009). Uniform bounds for norms of sums of independent 

random functions. Ann. Prohah. 39, 6, 2318-2384. 
GOLUBEV, Y. and Spokoiny, V. (2009). Exponential bounds for minimum contrast estimators. 

Electron. J. Stat. 3, 712-746. 
Kalinauskaite, N. (1966). Upper and lower functions for sums of independent random variables 

with limiting stable distribution. Litovsk. Mat.Sh. 6, 249-256. 
Kerkyacharian, G. , Lepski, O.V. and Picard, D. (2001). Non linear estimation in anisotropic 

multi-index denoising. Prohah. Theory and Relat. Fields 121, 137-170. 
Kluchnikoff, N. (2005). On adaptive estimation of anysotropic functions. Ph.d Thesis. Aix- 

Marseille 1. . 

Lepski, O. V. (1991). Asymptotically minimax adaptive estimation. L Upper bounds. Optimally 

adaptive estimates. Theory Prohah. Appl. 36, 682-697. 
Lepski, O.V. , Mammen, E. and Spokoiny V.G. (1997). Ideal spatial adaptation to inhomo- 

geneous smoothness: an approach based on kernel estimates with variable bandwidth selection. 

Ann. Statist. 25, 929-947. 
Lepski, O. V. and Spokoiny, V. G. (1997). Optimal pointwise adaptive methods in nonpara- 

metric estimation. Annals Stat. 25, 2512-2546. 
LiFSHiTS, M.A. (1995). Gaussian Random Functions. Kluwer, Dordrecht. 

Massart, p. (2000). About the constants in Talagrand's concentration inequalities for empirical 

processes. Ann. Prohah. 28, 863-884. 
QuALLS, C. and Watanabe, H. (1972). Asymptotic prpperties of Gaussian processes. Annals 

Math. Stat. 43, 580-596. 

Shirayev, A.N., Valkeila, E. and Vostrikova, L. (2002). On lower and upper function for 
square integrable martingales, in Proc.Steklov Inst. Math. 2 (237), 281-292. 

Spokoiny, V. (1996). Adaptive hypothesis testing using wavelets. Ann. Stat. 24, 2477-2498. 

Stone, C.J. (1985). Additive regression and other nonparametric models. Ann. Stat. 13, 2, 689- 
705. 

Talagrand, M. (1994). Sharper bounds for Gaussian and empirical processes. Ann. Prohah. 22, 
28-76. 

Talagrand, M. (2005). The Generic Chaining. Springer, New York. 



72 



TSYBAKOV, A.B. (1998). Pointwise and sup-norm sharp adaptive estimation of functions on the 

Sobolev classes. Ann. Stat. 26, 2420-2469. 
VAN DE Geer, S. (2000). Applications of Empirical Process Theory. Cambridge University Press, 

Cambridge. 

VAN DER Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. 
Springer, New York. 



73 



